I recommend "The Theoretical Minimum" books as a good starter to get you through to the level of mathematics needed to be able to read the quantum math and vector/tensor calculus used in field equations. Statistical calculus also.
Once you have these under your belt, comprehending the real physics textbooks and papers will be unlocked.
This stuff is not hard are impenetrable, but the language is if you don't know it. The language isn't hard or impenetrable. E.G. Vector calculus is much simpler that algebra. Just find a good book or teacher that doesn't blind you with procedure over concepts.