Right, they are saying it is the result of a pattern of behaviour. Like when you find ashes it is evidence of a fire, but that doesn't mean you need to have an ashes quota. It means you need to prevent fires.
By "people like you" I mean people who leap to unfounded conclusions based on their existing preconceptions. Take your comment "Old Fuddy Duddy. Get the thought police it's hate speech." No-one is saying it is hate speech. You made that bit up yourself, it's your own reaction, your own mind that thinks that way. They are saying it is discriminatory in the context of a PATTERN of similar remarks.
To win a discrimination case there needs to be a pattern, an on-going problem. A single remark is not enough if that's all it was.