For a regular school, particularly state school, then yes it gets stacked a lot by test scores and other academic indicators. The better you do academically, the more they are interested in you and the more money they'll try to give you to get you to attend.
However the "elite" schools have a whole bunch of good old boy shit going on. If you look at admissions in to places like Harvard you find that there are some legitimately top performers who come in, but a whole lot who are not and are instead connected some way. They are kids of alums, politically connected, rich, whatever. They are the "right kind of people" and so get the invite.
That's also the reason why parents want kids to go there is the connections. You don't get a better education at Harvard overall. Any university with a good program will do at least as well, and in plenty of disciplines there are schools ranked far better. However it further gets you in to the old boys club and gets you connections to people that gets your opportunities that would not otherwise be available later in life.