First, forget the notion that some people are unable to learn math. Some may be better than others, but like language, the human brain is wired to understand mathematical notions. The timing required to track the trajectory of a ball and catch it is a very sophisticated math problem that even very young children can master.
Second, remember that math is a tool we've constructed to solve problems. For most people, having a detachment between learning something tangible and learning a tool to solve it, short circuits the natural process of learning. Learning, or at least learning well, virtually always requires an interest or need in the subject matter.
So treating math as just formulas and rules and a vocabulary of terms to remember is a huge problem. To understand math, you should understand the underlying concepts and how they apply to the real world, and being able to conceptualize those ideas when you're looking at formulas, etc. A good teacher, math or otherwise, will implicitly to this for you. However, there really are very few good teachers around, either at primary or higher education levels. And colleges are increasingly money machines designed to crank students through the system, and even good professors succumb to the pressures of getting as many students through courses as possible, instead of taking the time to help people understand. After all, by college, shouldn't you be prepared to do all that work yourself? Or so goes the thinking.
Start with understanding basic ideas of linear rates, and find a parallel in real life, e.g., like driving on the freeway. See how the position and speed of your car on the freeway corresponds to a linear function that intersects the x and y axis of a graph. Then move to understand acceleration, and what that looks like on a graph. Then think of two cars in different places moving toward each other at different speeds, and how lines on a graph that intersect can help you understand when they will crash. Now what if one is accelerating? If you're not interested in cars and driving, pick something else of interest to you. At each step of the way in learning math, if you make sure you understand the underlying concept and how to visualize in action in day to day life, you will make learning much much easier.