The issue is really that of resources. You don't want to keep a standing army of the top-notch commandos just because the terrorists can analyze the vulnerabilities using the satellite imagery.
As it is, some parts of New Delhi look like a war zone, and I'd hate to see army posts at each intersection asking me where I was going just because they are probable security vulnerabilities. Pretty much the same way I hate it when someone looks at my email logs or the pattern of shell commands or prevents me from accessing certain commands on the system because they may be vulnerabilities.
I can't believe I'm saying this, but under certain (very) limited circumstances it's also about enabling civil liberties (freedom of movement and not being watched all the time) by hiding certain soft-spots that I wouldn't see walking on the street.
The rule on staying alive as a program manager is to give 'em a number or give 'em a date, but never give 'em both at once.