> Flashing or raping or molesting or peeping is a crime, and those people should be arrested, and will be arrested even without laws to close the "loophole" as you put it. In cases where such behavior isn't happening, then your only reason for making their bathroom use illegal is that it makes you feel icky, and that's just not a good enough reason to put someone in jail or subject people to harassment or other mistreatment.
Yes those things are crimes, but why give the opportunity? Why allow the risk? How easily can this be prosecuted after the fact? Can they find the perpetrator? What about the victim that has been assaulted? How can the victim be made whole?
If you remove some of the risk, the possibility, the opportunity, you can create a place of safety (which already exists in many cases), and if that comes at the expense of some guy's feelings that likes to wear dresses and cosplay as a woman then I think that is a fair trade. All these guys that want to use the women's washroom are only thinking about themselves. Their feelings, their 'rights'. What about the women? What about their rights? We wouldn't like to subject these people (women) to harassment or other mistreatment, after all.