Freedom of speech doesn't mean speech without consequences. See: slander. How is that different? If the first amendment is absolute, then we should let people lie and destroy other's reputation at will.
For every complex problem, there is a solution that is simple, neat, and wrong. -- H. L. Mencken