They get special treatment and financial support because they were, quite recently, the victims of terrible human rights violations; and in some cases, still are. Those violations have a trickle-down effect. The current generation, even if not victimized directly, is still disadvantaged by their ancestry.
Further, on the one hand, you suggest that our standards of decency are wrong-minded (that we should not provide support to these people) but on the other subtly judge their ancestors by those same standards.
Perhaps you just want to have your cake and eat it too, as long as the pigmented people don't have a slice.