Here's two reasons:
1. Helmets give the cyclist a false sense of security.
2. Helmets give drivers a false sense of security.
You may think  does not apply to you, and possibly it doesn't but people are incredibly bad at judging that kind of thing. It's very likely that you take more risks when wearing a helment.
The second point is far more important and it's not something you as a cyclist can do anything about. Studies have shown that cars pass closer and faster to bikes when the cyclist is wearing a helmet. On some subconscious level they see the cyclist as being less vulnerable and hence they drive more dangerously around them.
For these reasons I discourage my three daughters from riding helmets when they cycle and I don't wear them myself.
However, even if one discounted both these reasons, mandatory helmets are horrible on principle. Its my own life I may be putting in danger, so if you want to wear a helmet, go ahead, if you want to tell other people to wear a helmet, go fuck yourself.