I doubt that this is right. It's just a misconception to say that in a first-world society, fertility is pretty much the same thing as attractiveness. It's not. In fact, the people who are broadly judged to be most desirable - the people with Ph.D's, sixpack abs and fancy jobs - have fewer children that the average. A much stronger driver of first-world fertility in a place like NL is: Who's sloppy with their birth control, who's impulsive enough to think things like "Yeah, I should just have the baby!", who's someone that thinks that having a child is going to fix the problems in their relationship, etc.
For these fertility increasers to be correlated with height is just weird and hard to explain, but it's obviously real, so there much be some mechanism. But that mechanism is not as simple as "Taller men get more girls".