Smart optical nanomaterials, also known as responsive optical nanomaterials, can allow the modulation of their optical properties in response to external stimuli and thus hold great promise for emerging applications in military camouflage, display, sensing, counterfeiting, artificial intelligence, etc. Among them, magnetically responsive optical nanomaterials have attracted tremendous attention owing to their fast response to external magnetic fields, high sensitivity, full reversibility of tuning, and contactless control. During the past two decades, intensive efforts have been made to develop various types of magnetically responsive optical nanomaterials and explore their applications in diverse areas. These studies have enabled magnetic tuning of different optical properties such as structural color, transmittance, and localized surface plasmon resonance. In this review, we summarize recent advances in the developments of various magnetically-responsive smart optical nanomaterials. Mechanisms of magnetically tuning the structure of colloidal assemblies and the orientation of non-spherical colloids are first introduced. We then describe several examples of magnetically responsive smart optical materials based on magnetic tuning of a single optical property, such as the diffraction, transmittance, and plasmonic excitation. Recent breakthroughs in the development of smart optical materials with multiple magnetooptical responses are also highlighted. Finally, perspectives in terms of challenges and future research in this area are discussed.