Perhaps part of the reason you don't have to worry about it so badly is that numerous governments are constantly conducting investigations, surveillance, and operations to keep terrorism at bay. Nothing is 100% guaranteed of course. How much worse might it be if they did nothing? Much has been done to contain Al Qaeda and ISIS, or their supporters. It's a bit like saying, "Why am I paying my mechanic for all this maintenance upkeep on my car, it never breaks down". Is money being spent frivolously as well? It sure is, that's the nature of government services.
I suspect, however, physical damage and carnage are secondary concerns to the topmost levels of government; the primary reason governments want to squash terrorism is because of it's negative effect on Wall St. Markets are very high strung and touchy, and even a little terrorism goes a long way.