I work for a medium-large organization (a few thousand people worldwide, nothing like a Microsoft or IBM.) Both very small and very large organizations have problems. Small businesses are usually run by a tyrannical owner and their family, and all others are treated like "the help". Large organizations develop their own political infrastructure, and yes, they collect a lot of unnecessary employees. I'm not sure which is the bigger problem.
When things get too big, there are some people who get very good at either (a) hiding out and not doing a whole lot, or (b) taking advantage of the size of the organization and slowly building empires around themselves. I'm on a very small (way too small for the amount of actual, real customer work we do) product engineering team and am sometimes amazed at how easily some other groups within our company can just ask for and receive more headcount. Good politicians do very well in large organizations. In addition, there are simply a lot of jobs that involve processes that could be automated, but for whatever reason they're not. How many large-company employees do you know that simply take an input stack of work, perform some sort of transaction on it, and pass it on to the next person in the chain? A lot of this is probably holdover from when companies actually did have thousands of people manually processing paper and requests.
Also, in large organizations with long-term employees, it's very easy for the employees to get wrapped up in the organizational procedures themselves. I have a lot of friends who work for the state university system and in local governments, and they tell me all sorts of stories about people throwing fits over the number of sick days they have banked, etc. just because it's a very important part of their work culture. There's a lot of bureaucracy just for the sake of it, and long-term employees use "the system" to maximum advantage. The problem is that it distracts from the actual work that needs to be done.
I'm not really sure we _should_ get rid of every single inefficient position, for one simple reason...these office jobs keep a huge chunk of middle class with reasonable skills and medium levels of education employed. Take those out everywhere and suddenly millions of people start defaulting on their debts and the economy collapses. In that case, either (a) the economy reorganizes around a Star Trek The Next Generation model, or (b) we start seeing some really bad stuff happening in the near future. Losing manufacturing was bad -- imagine what happens when millions more have nowhere to go and nothing to do.
That said, try to get a bug fixed or feature added in Windows or Office...it's not easy and I think I know part of the reason.