It seems SpaceX is relying on a trial-and-error strategy during the development of the soft landing capability of their booster much more than they (or others in the industry) do for other components or capabilities of space launch or other aeronautical systems. I don't see (unmanned) rockets or drones being developed in this fashion. Even large rockets that can achieve orbit will normally be modeled, simulated and tested component-wise to the point that they will usually work at the first or second attempt when the entire system is integrated and tested for the first time. So why is this so different here? Is it just cheaper? Or is it actually that much harder to make the rocket land softly on its own exhaust jet than to make it go into orbit?
It's important to remember that the primary mission was a complete success. The Dragon delivered the cargo to the ISS and is awaiting trash and cargo to return to Earth. This was a post mission experiment meant to collect data. It's very common to completely loose a rocket in the early flights, but that's not what happened here.
SpaceX does what's called LEAN development, which is basically like agile software development. Really all development is incremental, the difference with lean/agile is you admit that instead of pretending that you can design the perfect solution from the start. SpaceX has a huge computer cluster and they model the hell out of everything they do. Then they try it to see how it works in the real world, measure the results and make improvements. The experiments are always done after stage separation in a way that collects important data without putting the mission as risk. You can call that trial and error, but that does the process a disservice.
There have been experimental rockets and landers that land vertically, most notably the DCX. But no one has reentered a first stage of an actual in service rocket, the previous vehicles have always been test platforms and never accelerating to launch vehicle velocities nor going to launch vehicle altitudes. NASA has flown aircraft to collect data from earlier SpaceX missions because no one else has EVER controlled a first stage's return to earth. (Shuttle SRBs were not controlled, just big steel tubes falling from lower and slower than the F9.) The first stage is a long cylinder with blunt ends and it reenters the atmosphere at hypersonic velocities. On top of that, it's a super light weight and fragile airframe. Just getting the thing down to terminal velocity in one piece is a big deal.
The LEAN development model is less expensive than the classic approach. It's also faster and yields really good results. You learn about problems sooner and don't bake them too deeply into your design. Look at it this way, the closest competitor to SpaceX in developing a reusable VTVL rocket is Blue Origin, started by Jeff Bezos. Blue Origin started with more money than SpaceX and before SpaceX. SpaceX is delivering cargo to the ISS, and about to test the Dragon V2 abort system in preparation of flying astronauts in 2017. They are also self funding the development of a much bigger reusable rocket (slightly bigger than the Saturn V). They are doing all of this while providing the least expensive launch prices in the world. Less expensive than Russia. Meanwhile Blue Origin hasn't even reached orbit. They aren't even trying to reach orbit, they are still developing a suborbital rocket, even though they have a number of experienced engineers that worked on the DCX. Oh, to be fair, Blue Origin is developing an engine for use by ULA (and Blue Origin) and doing some work on a man rated capsule. But nothing is anywhere close to flying.
