Like many things, the theory is great, but the execution isn't necessarily so. I agree that the "needs to not be big enough to corrupt the entities that should be overseeing/counterbalancing" part, and would add that it also works best in areas where other competition exists. For instance, the government (or some level thereof) is the only "consumer" for private prison services. There's no outside expertise specific to this area, save to the extent that we create it by privatizing an inherently governmental function. On the other hand, things like construction are commonplace, because everyone uses it, and it makes no sense for the government not to rely on the same private companies that everyone else does when they want to put up a new building.
To toss in a pair of anecdotes from my personal experience:
When I was in the military, I saw what I'd consider a "good" example. Instead of having soldiers run the dining halls during basic training, rotating in a new group of wholly untrained people every 1-2 weeks, and taking that time away from training (at severe cost to the government, who was paying not only salary but food/housing/etc), they contracted it out to a food service company, that brought in regular workers at prevailing wages. This not only cost less than using trainees, but provided a lot more consistency in terms of service (which can be very important in terms of proper food handling).
On the other hand, I later got to see what happened with contracting in the Intelligence community. Now, this was a while ago, but the situation still hasn't changed at all. Essentially, the government was/is paying more to bring in contract workers. The supposed advantage was that they were easier to hire/fire, but in practice this rarely seemed to come into play. What was worse was that the vast majority of these people were former government/military personnel, because pretty much the only place to get training/experience in that field is... ...working for the government. It basically amounts to creating increased competition for the same pool of workers, causing the cost for those workers to rise, which winds up costing the government far more. To some degree it's good for the workers, since they have more options for more money, but most of the extra money isn't going to them - it's going to the companies, some as wasted duplicative overhead, but mostly as profit.