Working for government agencies is widely reported to be sucky for a variety of reasons: lower pay than the private sector, heavy bureaucracy, political infighting, mediocre employees. It's just unattractive all around.
I'm a state employee (state university), not a federal one. But here's my counterpoint to your (valid) points.
Pay is definitely lower, but the benefits (vacation days, retirement, etc.) are often better. The overall workplace pressure is often less. And, as in the private sector, the "quality" of your coworkers really varies from one group to another - so it's not a given you'll only be working with "mediocre employees".