If you earn only $20k per year, you receive government assistance for food and medical needs.
That's beside the point though -- it's tough living on that amount of money. Hell, why don't we just take another $20k from the rich person and just give it to the poor guy earning $20k? Now he's got $40k and the rich person is still rich. But it's tough living on that amount of money too ... even though we just doubled it. So where do you draw the line?
At some point you have to realize that the person earning only $20k per year has made decisions throughout their life which lead them there. THEIR choices led them there. We don't live in a socialist society; the fact that life is hard for person a doesn't give you the right to take something from person b (whom you perceive to have it easy); all men are created equal. The goal is to ensure that people living below the poverty line don't die of sickness or starve to death, not to make their life easier for them because they regret the decisions they've made.
At the end of the day, no matter what you make, if you made x% more things would be easier: you could afford a nicer car, a bigger house, better vacation, take your wife out for a nice dinner, better college for your child, etc. You certainly don't NEED those things, but you want them. And then you find yourself looking at the guy earning over $500k and think to yourself "gee, he's got a lot of things he doesn't need; if only he was a bit less greedy and shared it with me/that poor guy/whomever, I could get something that I WANT."