By definition the private sector has to be more expensive at achieving a goal than the public one.
Not at all. The public sector tends not to care about costs, since they take the money more or less by force (implied force, if nothing else), and they have little to no threat of competition to force overheads to remain low. And of course one government providing a service for money to another government also has the motive of profit, making this situation more or less the worst of all possible worlds. In a theoretical optimum world, public sector would be by definition cheaper. Unfortunately, we live in the real world, which isn't always quite so nice (a tiny snag that many political philosophers/economists/et al often overlook).
In this case, for example, SpaceX is attempting to lower costs through a practical reusable design, whereas the Space Shuttle (in practice) ended up raising them considerably, despite being reusable, due to a number of ridiculous design constraints enforced on it by various government interests.