I come from the UK and personally find mathematics pretty difficult. I can work through problems on paper but my mental arithmetic is atrocious. By the time I two operands and an operator in my head and have broken up the problem into a simpler problem, I have forgotten the original two numbers...
That said, mathematics should come the more you practice. I like to blame the school curriculum -- it is shit. The only reason why I am valuable is because I acquired computing skills playing on computers as a child.
I'd like to blame mathematics textbooks but I cannot. My generation and a few before me have lost the willpower and motivation to actually study and learn things properly. Our education system does not really promote mathematics that well. My school staff was rife with young twenty somethings fresh out of university with no real ability to teach...
Teaching has lost its respect and professionalism in the UK too. Add to the fact it became okay and even cool to be ignorant in modern culture.