An identity card system is not the same as a security system; in fact, the governmental standards for single ID specifically preclude its use as a privilege system. The proposed ID's have the capability of identifying the individual as a government employee/contractor, and, combined with some other security systems, will grant or deny access to different assets (buildings, computer systems, etc.) But the card is not a magic golden ticket to let you do whatever you want. Just because I'm a fed and can walk into a Federal courthouse in Dallas doesn't mean I can walk into the Federal Reserve Building in New York.
That said, though, most government employees can enter most non-sensitive federal buildings with the ID's they already have. Federal employees from lots of different agencies need to do business with other agencies, and there's so much crossover that it's not practical for building access to be defined on an employee-by-employee basis. It's easier to say "no" to those few high-security buildings and areas than it is to say "yes" to the umpteen zillion buildings with minimal security concerns.
This is really a non-issue... for physical access this system is just a consolidated, unified approach to a system already in place. For electronic access (i.e., computers) this is just a varient of the secure smart card system already used by numerous private industries. The only real "universality" is that the ID's will look the same and the tech will come from a few contractors, instead of many. It will certainly save money as it replaces smaller, localized contracts that do the same thing. But it will not open the government to a single point of attack as many posters are implying. We will not be worse off than before, and will probably be better off.
Paranoia is fun, but this is not a big deal.
