Let me see? Space is quite big, unimaginably big in fact, and it is very likely there will be insane amounts resources out there on mostly dead planets, with no ecological harm done to anyone. Not to mention we can only examine a insignificant fraction of it in any detail, i.e. not through inferring information from specs of electromagnetic radiation.
Yes we are a long way from exploring even our solar system in any great detail or harvesting any resources, however if we do not make a start, because we keep saying whats the immediate payback? We will never get there. We also are currently benefiting space technology such as communications, GPS, and the government being able to better spy on us, oops the last one might not be a benefit.
As for people suffering, yes there are, but it isn't through lack of resources, it is through our greed, fear and hate. In the US 40% of food produced is waste. Obesity is a problem. Obesity kills 3 times as many people as malnutrition worldwide, (http://www.telegraph.co.uk/news/health/news/9742960/Obesity-killing-three-times-as-many-as-malnutrition.html), OK the malnourished are probably more likely to young children, but still we clearly have enough resources. There are 7 billion people in the world, if a few thousand concentrate on building rockets, it is not going stop the rest of us coming up with a solution. In order to fix suffering we need social/political solution not a scientific one.