Limited by whom? Stop using the passive and name the entity that will stop the government from simply assuming more power yet fails to use dictatorial power itself. A civic-minded populace can do it, sure, but it also tends to extend the role of the government to help further various agendas, resulting in Nordic welfare countries at best and the corrupt American mess at worst.
That's a bit cynical and narrow. Prior to the destruction of federalism in this country, sovereign states were capable of limiting federal powers, and the courts tended to err on the side of the people, rather than the legislature, when it came to adjudicating the constitutionality of laws. That was all turned on its ear during the FDR administration, with Wickard v. Filburn being one of the more egregious examples.
Also, in your model, who will build all the infrastructure that's enormously important to the society but can take a decade or more to begin turning a profit? The free market is notoriously bad at long-term planning. For that matter, who will force the re-internalization of externalities (such as pollution controls), without which the free market gets twisted in an outright psychotic manner (whoever cares the least makes the most profit)? A strictly limited government cannot do either of these, nor can it deal with whatever future issues come up unless its scope can extend (in which case it will extend right back to where it's now).
Why is it that every big-government proponent equates "limited government" with "nonexistent government." You will be hard-pressed to find anyone, aside from a libertarian who is so pure that he could be described as an anarchist, that would dispute that government has legitimate roles, your aforementioned examples among them. This is a common straw man, and it does nothing to bolster your point.
The right refuses to accept the fact that power cannot be destroyed, just shifted around. Strip the politicians of the power and the wealthy will wield it directly, like they did in the past. Big powerful government is the only institution standing between us and feudalism.
This is simply false. The US government, as originally designed, was unique in that it offered a way out of feudalism. It is only through government fiat that the wealthy could wield any power at all. They are inoculated from liability thanks to corporate law. Regulation and taxation has put up barriers to entry in nearly every market. Henry Ford was able to start his automobile company as a middle class worker. Today, with the exception of building out a web product like Twitter or Facebook (expect that particular area of opportunity to close once the US sees fit to slam the Internet with regulations), that notion is all but dead. To emulate Ford, one would have to navigate hundreds or more regulations, and submit permit after permit, just to get off the ground, and if he would manage some meager successes, existing corporations in that market would wield the most esoteric of rules like a cudgel in the court system, until he was pounded out of business. We have a system of megacorporate hegemony, enforced by US law, under which most of the people in this country serve as worker drones. This is simply modern feudalism. The lords are found in the Fortune 100, they carry water for their king in the US government, and their employees are serfs.