that's an old and tired argument, which sounds superficially reasonable - until you realize that civilization and progress never worked this way.
you could have used that same argument for any human advancement.
do you really want to spend money on chemical research / the Panama Canal / railroads / telegraph / cables under the sea/ etc, while there are people starving to death or dying of flu or lacking basic freedoms?
none of those things were ever a priority when money was spent on them throughout the centuries .
yet, without those, we would be living today in a world where millions more would be starving and be less free.