At best, the intent of hate speech is to marginalize and silence others, at worst it's about permanently silencing others.
Free speech has no room for hate speech because allowing hate speech to flourish means many will avoid speaking up since they feel intimidated.
There's a reason that Poppers paradox was written in 1945 which it says "unlimited tolerance must lead to the disappearance of tolerance... We must therefore claim, in the name of tolerance, the right not to tolerate intolerance".
It's the same with free speech and hate speech, either you support freedom of speech without accepting hate speech, or you support the idea that it's okay to silence others through hate speech and intimidation.
Anyone has the right to speak their mind but they can't force themselves and their speech on other people.