This is because racism is in fact (somewhat of) a one-way street. White people have traditionally been (and still are, by and large) in the privileged position in our society. Thankfully, it's not true to nearly the extent that it was decades ago (although hispanics/arabs seem to have filled that space in America), but ignoring the fact that black people are a historically oppressed class and then whining about how white people have it so bad is either ignorant, stupid, or both.