Programming isn't, strictly speaking, math. Not like calculus or algebra is math. It's something else entirely.
I have a rather annoying learning disability, and have suffered from it since I was in first grade: I have a great difficulty memorizing things. Now, even with that rather annoying problem, I managed to be near the top of the class in both grade school and high school. What happened is I learned early on how to analyze the patterns behind the subjects I was expected to memorize and when test-time came was able to reconstruct most of the required information based on those patterns. Perfect example of this is spelling. I figured out what phoenetic combinations ruled English, and used them to pass tests. Armed with those rules, I only had to remember one or two odd words per week. In history, I found myself being able to understand and explain historical trends and influences. Names and Dates, on the other hand, gave me (literally) a headache. Simple math was easy (just a small number of consistent rules, after all). Sort of. Word problems in math, on the other hand, were difficult at best.
After I graduated college, I bought myself a Commodore 64 (they were just on the market) and dove right into both Basic and Assembly language. Both were easy for me to analyze and comprehend. Logic may be a branch of mathematics, but to me it's more obvious than anything in the math textbooks. Calculus is (for me) as obtuse and illogical as it gets. "If a water tank with a height of 15 feet and a circumference of 20 feet is 2/3 full of water, with a 1/8 inch hole a foot above the bottom, how long will it take the water level to drop to the hole?" Just plug the hole, whydoncha? Jeez!
Interestingly enough, I also have problems with computer word problems. Just give me the code (any code -- C, C++, C#, pascal, basic, assembler, SQL) and a little time, and I'll tell you what's causing the bug you're seeing. Now, after all these years, I'm the one everyone comes to when faced with an intractable computer problem that no one else seems able to solve. And I still don't like math.