Simple: you have to perform your launch from a balloon. You have to cram any support gear that would normally go on a pad into the balloon, and hopefully work out some way to get it back. You can't do any test fires or pad aborts, you're committed to either a successful launch or loss of vehicle and payload once the balloon leaves the ground. You can only launch when the weather's good enough to inflate and fly the carrier balloon, which is far more sensitive to bad weather than rockets. Even gigantic balloons would be limited to tiny rockets...for example, the Falcon 9 v1.1 masses over 500 metric tons, and SpaceX has bigger and more capable rockets planned.
And the big one: if you're going to orbit, there's just no reason to launch from a balloon. Gaining altitude is a small part of reaching orbit, balloon launch is really only useful for small suborbital rockets where the starting altitude more or less directly adds to the peak altitude reached. And even then, for rockets like Starscraper, there's the above limitations with weather, mass, etc, and the additional added trouble with recovery due to the drift of the balloon before the rocket launches.