I'm getting my MSc in the same field (Score 1)

I retired from a thirty year career as a software engineer to do the same thing you are doing. I am in my 4th semester towards my MSc in astrophysics and astronomy with grades of High Distinction each semester. If you know math up through calculus and advanced differential equations, then the physics will come easily! Your first semester may require some spherical trigonometry (which even most math majors aren't familiar with), but essentially you've got it made with your math background. I'm familiar with practically all of the books that others have recommended to you in other comments, but I've found that the most relevant are the David McMahon "Demystified" books such as Relativiy Demystified , Quantum Mechanics Demystified , etc. These books assume you are familiar with advanced math but not necessarily physics (though be warned I have found some mistakes in these, but these errors are easily recognizable). I honestly think you don't have much to worry about given your math background.