That's the problem, a lot of guys suck it up because they don't want to be seen as week.
I just went through a he/she said situation at work. Result -"Even though you did not make any physical advances, suggestions, requests, etc, towards anyone, some people (who we can't name) say you talked explicitly about a date you were on so we are going to move you to a final counseling. Yes, we know you don't have a history of it, but why would a group of 3 friends lie?"
At that point, any mention of how those 3 people have made my work life a living hell for the past year sounds like me trying to lie about them to get myself out of trouble.
Off topic, going forward, how do you prove you did not do something?