First, you need to know (or remember) that huge corporations (be they defense contractor, Oracle, Microsoft, Google, whomever) are often a conglomeration of previously small companies. The company I've worked for has changed names four times as it was bought up repeatedly (twice in a three-month span one year) and is only most recently called "Northrop Grumman" but, for the most part, I still work with and for the same small group of people I hired on with nearly a decade ago. Yes, corporations add capabilities when they see opportunity. Who wouldn't?
Second, depending upon the work you do, adding all of the additional infrastructure required to meet the various regulatory requirements of a government contract is non-trivial - security clearances alone, if required, can be a nightmare. The company never says "Hey, I want to add more costs to my bottom-line and reduce my profits". Those bureaucratic requirements are driven by the government, not the contractor.
Third, often times the contracts awarded by the government require or strongly encourage the Bigs (like Northrop Grumman, Lockheed Martin, Boeing, etc.) to hire "Smalls" - smaller, perhaps more specialized corporations, that would not otherwise be able to get involved in these contracts. "Disadvantaged" small businesses, those run by minorities or other protected classes, are also highly sought after by the Bigs in order to meet various participatory quotas, etc. This type of thing allows the Big to address the regulatory and management issues while funneling funds to Smalls who might do much of the work. You learn after awhile, at least at Northrop Grumman, that you are an integrator first and a developer second - if you can reuse something someone else has built that is *much* better than building it yourself (you know, that whole "reuse" idea that we've all been chasing after for the past 50 years).
All of that aside, a huge amount of costs are associated with government bureaucracy. The profit margins on my contract, for example, are *limited* to 8.5%. No matter how much I spend, I'm only going to earn 8.5% - that profit margin is ridiculously tiny when you consider what a firm operating at commercial rates is going to make profits wise. "Oh...so you'll just drag it out so you make more money." Ha! Sure. The project drags out...but I can tell you from 15 years working with the US Government, it drags on and on not because I really want to keep working on the same stinkin' thing (redoing it over and over) for my own giggles and grins but because the US Government is a huge bureaucracy and it takes forever for them to make a decision on anything. Need clarification on a feature request? Well...first we have to work that through the Government Program Management Office (PMO) who oversees your project, then they need to potentially track down user-representatives, convene a meeting, possibly do a usability test and/or request a conference, get multiple disparate agencies who are going to use your tool to agree to put aside their differences and unique business processes, etc., etc., etc.
Meanwhile, the team is being held to an unrealistic schedule set for political reasons. To minimize risk of schedule slippage, you make a decision and press on accepting the fact that you may have to rework the feature you just developed. The government entities that are closest to you are just as frustrated as you are...and they know they can't let your team go onto other projects because then they lose the people who understand the project and its history - who have the requirements and design knowledge to meet the needs of the customer so they keep giving you money to keep your team together. You do your best to catch up on the copious amounts of documentation the government requires (for my 450K SLOC system, I'll probably produce 5K-7K pages of documentation between various specifications, testing plans, installation guides, user guides, etc.) and then suddenly some decisions are made and you sprint like mad to get those implemented to attempt to stay on schedule and then...you find the next thing you need an answer to...
Yeah, it is easy to blame the so-called greedy defense contractor from the company that "didn't traditionally do IT" but you certainly are missing the point that that company is made up of individuals who, possibly like you, have always done IT. Its not like they grabbed the guys building the warships on the coast and said "Hey, Bob, yesterday you were a ship-builder; today you're writing software and integrating communications systems. Welcome to your new job!".
I'm not saying that defense contracting companies are hurting or anything and that they are all managed by a bunch of nice guys and gals and oh-shouldn't-we-just-give-them-the-benefit-of-the-doubt but if you think those high costs and schedule slips are just them exploiting the government or some indication of ineptness, then you need to spend some time (more time??) working for a firm contracting with the government. It will quickly become apparent to you that the government is at least equally culpable in budget and schedule delays.
A statement like "...because when I think on-time, on-budget I think big government". That would be closer to the truth, at least.