Your theory would make sense if the majority or even a significant number of bathrooms had some kind of controls over who could enter them based on apparent gender. But for the vast majority of bathrooms in the world, or in the US, or in fact in any place you can name, there are no such controls.
As such, the idea that not letting people with penises [claim to*] be women and use women's bathrooms protects women from being sexually assaulted is, frankly, spectacularly stupid. Men dressed as men can just walk into them and rape women. In fact, it happens dramatically more frequently than a chick with a dick doing it.
* It doesn't matter which for the purposes of this argument, and I'm not interested in having that debate in this thread