I was fortunate enough to listen to an hour long debate about ten years ago between Neil DeGrasse Tyson and Bill Nye on this subject at the National Science Teachers Association Conference. Tyson was on President G.W. Bush's manned spaceflight council and made the same basic argument you did, while Nye made an argument very similar to TFA - science now, humans later. At the end of the debate there was no clear "winner." I think most of the 300+ of us in attendance just walked away wishing that we put more money into both types of programs as they both have great value.
It always just comes down to money, and this is part of a much larger issue IMHO. Our government is not funding basic science at anywhere near the level they should. Everything is left to business, and as a result the vast majority of research being done is focused on immediate return (and profit), not on long-term gains.