Whilst I understand your point, your solution is based on the premise that intelligence is innate and heritable. Although this may be partly true, there is considerable evidence that intelligence can be acquired by the correct type of mental training - directing children early into certain schools and therefore certain career paths ignores the brain's ability to change itself. If the brain is truely neuroplastic, categorisation then becomes based on which students have trained their brains better for the tests - or, more likely, which parents have had their kids tutored for the tests.
Where I live, it is common for children under eight to be taken to after school tuition sessions run by private companies - which means categorising students based on test results is as much a test of their parents' desire for their child to do well academically as it is a measure of their actual ability. This will entrench the class system further, rather than allowing brighter than average underpriviledged kids to do well. I admit that the outcome would be considerably better for a genius from an underpriviledged neighbourhood, but a public education system should be designed to benefit the majority of students, rather than the one in a million.
Diverting students to selective schools will also increase the consequences of external factors which cause children to underperform at school, such as divorce, poverty and plain lack of sleep. These effects will be more obvious in younger children - although a seventeen year old may have the mental discipline to study despite the situation at home, a stable home environment is a prerequisite for a thirteen year old trying to do homework. As a stable home environment is more likely in a middle class household than in a poor one, lower class kids will be diverted away from the academic schools towards ones with designed to develop them for roles "appropriate" for them as people of "normal intellectual means".
It also fails to account for differences due to age; children in a year level can vary in age by at least a year (depending on local rules), as children born early in the year are advantaged. Differences are most pronounced at puberty, so categorising adolescents and diverting them to different schools at this time will penalise and reward individuals on an essentially arbitary criteria.
I agree with your broader social point however - people should not have to be exceptionally mentally talented in order to lead a dignified life.
As a disclaimer: I have never been to the USA, and do not have first hand experience with its public schools. Ironically, I first went to the Australian equivalent of an Ivy League school before transferring into a selective entry government run school (I suppose you could call it a "gifted" school), so my experience at school resembles what a bright student would experience under your proposed system. In that system, certain schools are renowned for teaching students to maximise their scores in the final exams, instead of teaching them to understand and learn - so if a student wants to go to university, high school is as much about playing the system as it is about ability and learning. This shows up at university; large numbers of students struggle and drop out in first year because the emphasis is on understanding rather than regurgitating pre-prepared essays.