No so. Throughout most of history, mathematicians did not have the luxury of pandering to nationalism, militarism, pacifism or other temporal concerns. The numbers of mathematicians were so low that from the very earliest days mathematics was an international scholarly activity.
While it is true that mathematics was employed by engineers and others in many applied fields, mathematics itself has never been subject to restriction or exclusion on the basis of its applications. The applications themselves perhaps, but never the mathematics. Even in the Soviet Union, mathematicians were free to research and publish as they pleased.
This, like so many things in science, has changed in the post war, "Big Science" era. We are now in a situation where ~1% of all mathematicians worldwide are employed by one organisation -- the NSA -- and the issues surrounding this organisation may yet lead to a wholly unprecedented crisis within mathematics, concerning what we should/shouldn't not work on -- or for. If we end up in a situation where certain branches of mathematics become restricted or prohibited in any way, then mathematics will have crossed a particularly dangerous Rubicon, and with it so will Western society.
As much as I don't like what the NSA is doing, the problem is with that organization, and not the tools, disciplines, or mathematics being done there. I for one am not willing to uproot millennia of mathematical traditions and precedent because one foreign power has allowed its spy organization to run out of control.
I note that the great French mathematician Alexander Grothendieck, effectively retired from mathematics in protest at, basically, the Vietnam war. Some view this as a powerful statement of principal, but I don't accept that mathematicians direct themselves according to events in the United States or any other country. Mathematics is an international, long-term and now global activity and that should not be compromised because of the likes of the NSA.
P.S.
If you are a mathematicians and you do want to do something about the NSA, please consider designing distributed secure browsing/email/DNS/messaging/hosting systems or contributing to their design. That will do far more for the world than fragmenting mathematics ever will.