Well, I think this is one of those cases where there's an umbrella rule that serve purpose, but which might also have sensible exceptions. In this case the rule is that selecting embryo sex is something that ought to be discouraged.
There are lots of reasons to discourage selecting offspring sex, some of which a reasonable person might disagree with. For example some would object that it's playing God. Others might say say that it's wrong to value persons of one sex more than others. I don't find those particular objections compelling, but one thing I do find convincing is that changing the almost 1:1 balance of reproductive age men and women could destabilize society in various ways. But note that under that particular objection we could certainly tolerate exceptions that are relatively rare. For example the slight discrepancy between the number of males who identify as gay and the number of females who identify as lesbian has no practical impact on straight people. Clearly an exception in this case would only affect a handful of people and is not a concern under the demographic balance argument.
The knee-jerk controversy that follows any proposal to do something which as a rule of thumb is frowned upon does serve a useful function. Because of confirmation bias, people tend to be blind to unintended consequences of things they've decided to do. Making them address those consequences is, within reason, a good idea.
Anyhow, that explains why restricting this to male embryos is more controversial than allowing either sex. Doing it at all is controversial because of the burden a failure could put on future society and the potential suffering it could cause.