The solid-state refrigeration technology is expected to replace the traditional gas compression refrigeration technology because of its environmental protection, high efficiency, and energy saving characteristics. Among various competitive solid-state cooling materials, Ni-Mn-based ferromagnetic memory alloys have received widespread attention due to their multifunctional properties such as magnetocaloric effect, elastocaloric effect, barocaloric effect, magnetoresistance, and magnetic field induced strain, etc. In recent years, materials engineers and scientists have carried out a series of in-depth studies on the thermal effects of Ni-Mn-based magnetic shape memory alloys and have obtained numerous research results. However, the poor toughness and cyclic stability greatly limited their practical application in solid-state refrigeration region. This article summarizes the influence of element doping, microstructure design and size effect on the strength and toughness of Ni-Mn-based ferromagnetic shape memory alloys, where the mechanisms are also summarized. The pros and cons of different methods in Ni-Mn-based alloy toughness enhancement are compared. The main problems existing in orbital hybridization methods are analyzed. The main research direction of Ni-Mn-based ferromagnetic memory alloy is prospected, which has important theoretical significance and research value for promoting the application of NiMn-based ferromagnetic memory alloy in various fields.