What you say is true, but "not being the most brilliant people I know" is not necessarily "failure", and definitely not "the floor".
I know a bunch of really brilliant CS people as well. Almost all of them graduated with CS degrees and doing quite well. There are different kinds of "smart". The person who becomes a competent sysadmin at 16 is different from the one who drops out of college to found a successful startup, and is different from the one who gets a CS PhD and then a researcher job at Princeton.
The 16-year-old sysadmin probably can't do my job, at least not as well as I (software engineering), and I probably can't do her job well either.
There are different paths to "success", although I must agree that the value of getting a degree is diminishing every day, it's not necessarily a bad thing to do. It really depends on the circumstances of each case.