There's no reason why Python (compiled from Rust) would be any slower than Python (compiled from C). Both languages compile to native executable code, and both have good optimizers available.

There's nothing wrong with knowing your own limitations and acting accordingly. Had Mr. Zuckerberg had that sort of self-knowledge, he might have avoided lighting 80-billion-with-a-b dollars on fire and renaming his social media company after a technology that nobody wanted.

That is true, but it's a little more complicated than that.

The average driver is pretty good, 99.9% of the time. The other 0.1% of the time, they are tired, or angry, or momentarily distracted, or just looking the other direction from where they need to be looking under those particular circumstances. It's those occasional blips of inattention that cause lead to most accidents occurring.

Yes, I agree. The problem with thinking "this bug seems harmless, because I can't imagine how anyone could exploit it" is in the "I can't imagine" part; my imagination is limited to what is covered by my mental model of how computers work, but an attackers' ingenuity is not.

In particular, the C/C++ optimizer is a devious beast, and will exploit any opportunity to make the code more efficient, even if that means doing things that are wildly unintuitive to a naive human reader -- and it sees any instance of undefined behavior as an opportunity to exploit.

(C) is the only one of those the AI companies are really concerned about -- they want to stave off model collapse for as long as possible. The problem is that AI generates so much content that the ratio of AI-generated data to human-generated data keeps rising, and sooner or later there simply won't be much human-generated data left to anything to consume. At that point they'll either have to figure out to ingest AI-generated content without causing degradation, or they'll have to accept that AIs have gotten as smart as they will ever get, and it's all downhill from there.

At present, no. I've done a number of experiments with LLMs' capabilities with generating fiction, and while they are impressive for a beginner writer -- because fo the lack of spelling and grammar mistakes, they are nowhere near being able to write competent fiction.

This is because they don't model reader or character minds, they're based on word probabilities, like a predictive text keyboard. The engineers can compensate for for this to a surprising degree by feeding it data to recognize specific situations (e.g. the car wash is 100 feet from my driveway; should I drive or walk?). But this is turd-polishing.

AI technology practice isn't foundational. Critical thinking is. The closer AIs come to being able to do something that looks like them doing our thinking for us, the more capable we have to be at thinking without them. A human who can't outthink an AI adds nothing to an AI-human partnership.

This proposed school is a typical scheme by people who think because they have been educated, they understand how the education works. This is like thinking because you eat, you know how to cook. Project work in education is nothing new; it goes back well over a century. And in real life, you may get a STEM degree, but your real education in engineering comes from real life work. But that doesn't mean classes in calculus and physics don't do anything for you.

Or English classes, for that matter. Nobody doubts the value of project work in education, particularly *androgogy* -- the education of adults. But for *pedagogy* you have to establish foundational skills as the student matures intellectually and socially. Entering the high school years, a typical student has had *no* training in critical thinking skills. To develop critical thinking skills in an area, you need both general skills (epistemologicla literacy: research, location of orignial sources, dealing with conflicting evidence) and domain specific training in that area. In other words, *liberal arts* training is foundational in an AI-saturated world. "Liberal Arts" -- from *artes liberales*: literally the skills necessary to be a free person.

But I think these people have at least got the problem right: as AI drives the cost of elaborate projects down, we all have more projects in our future. Big, elaborate, faultlessly plausible-looking projects which may or may not have actual value.

Where they are completely wrong is the solution, which is to organize education around practicing using AI to make impressive projects. In a word of AI-enamored dunces this will put you at the head of a very sorry class of ignoramuses. The answer is to build fundamental cognitive and intellectual skils at every stage of a student's development. Project work is surely part of that, but it's just a technique, not a solution.

In a nutshell, the academy is training students to be the next step down the evolutionary scale from code monkeys. More like the pigeons B.F. Skinner trained in WW2 to peck at a screen, in an attempt to create a primitive guided missile without computers. They want to create "main characters"; and they'll probably create a few. There are always a few natural autodidacts in any moderately large group. But what they are set up is to produce a bunch of fragile, economically privileged narcissists trained to cover their lack of education with impressive-looking AI slop.

It's really the same problem that we see with quantum physics -- the subject matter is esoteric enough that common sense can't be used to determine whether what you're listening to is really advanced theory or utter BS (or some combination of the two); they both sound the same, and the situation isn't helped by the presence of large numbers of people who think they know what they are talking about but don't. :/

I think the distinction they are making is between technology that kills people, and technology that entertains people. If someone fitted actual weapons to a Doom engine, they'd probably object.

Let's make the hopeful assumption that the superintelligent AIs will not "go rogue" in any way, but will correctly try to carry out their tasks as assigned by the people who designed and trained them.

Pretty much every major nation has military AIs, at this point. So the Chinese AIs are working hard on how to defeat the Americans, and the American AIs are working hard on how to defeat the Russians, and so on. In the final analysis, no matter where you live, there is probably an AI that has been explicitly tasked by its operators with figuring out the most effective ways to do you in. Not because they've "gone wrong", but because they're working as intended.

The size of the universe is increasing...
but the rate at which it is increasing is decreasing...
but the rate at which the rate is decreasing is increasing...
but the rate at which the rate of the rate is increasing is decreasing...
but the rate at which the rate of the rate of the rate is decreasing in increasing...
[... and so on, ad infinitum...]

People commonly overestimate the possibility of "all humans dying", and simultaneously underestimate the possibility of "billions of humans dying".

People are like cockroaches that way; a disaster could wipe out 99% of us, and there would still be millions of us left to repopulate afterwards. To truly wipe out all humans would take something more severe than the asteroid that did for the dinosaurs.

Whoever set up its reward function, really. Could be any of the above. The technology doesn't care, it only follows its algorithm.

It's a way of reframing sloppy practices as a signal to your investors that your software has become powerful. Most of the horror stories about AI showing signs of becoming hyperintelligent are coming from the companies themselves, which should tell us something.