I have two resumes in front of me. I need someone who can write some fairly complicated software. Are they writing the kernel to an operating system? No. But they'll be making complexity decisions between a server and a client. Not exactly new or novel but important to me and my clients.
So I look at one resume and the guy has suffered through integration by parts, linear algebra, differential equations and maybe even abstract algebra. The other guy went to a programming trade school where those are not taught. The trade school likely taught inheritance, pointers, typecasting, and all that good stuff just like the Bachelor's of Science degree would.
Now do my solutions need integration by parts, linear algebra and differential equations? Absolutely not. But if I'm going to pick between the two, I'm going to take the applicant that solved more difficult problems in order to make it to a class. Few people actually care about those concepts deep in their hearts -- and I'm sure neither of my prospective employees did. But in that same vein, no rational developer is going to care at all that my client likes to be able to drag and drop files instead of doing file navigation to find the files he wants. But I want the applicant who's going to do the inane stuff that he doesn't personally view as important.
Challenge yourself. Take the math courses. Take the logic courses. Take the statistics and combinatorics courses. Take the finite automata courses. Prove to yourself that there are no obstacles in your way. They are a great expense of time now but they are a huge investment in yourself -- no matter how pointless they appear to you.
If I had understood what I was doing, maybe I wouldn't mind so much.
You should attack this problem two different ways: 1) increase the amount of time you allot to your own personal enrichment in these topics/courses (three hours is very little time if you are approaching new concepts in math) and 2) seek outside instruction as it's also possible you have a professor who doesn't understand what they're doing either (the teaching, not the subject matter).