It's more like unpredictable software. I'm not saying we burn it, I'm saying don't give it agency. Only allow it to advise, not take action. I have experimentally used it myself on code, but did not allow it to commit to the repo, only show me it's proposed changes. It even proved somewhat useful at that, though I didn't apply it's changes, it did inspire a similar approach that I implemented and committed.

That a Teflon like surface takes part it? That is new ground.

A key to Capitalism from Smith himself is that it requires strong regulation to keep the playing field level. Smith also warned that charters must be given out exceedingly sparingly and the resulting corporation kept on a tight leash and held strictly to it's charter.

I guess we really blew that one!

But I rather doubt anyone involved in MAGA or others shouting "deregulate all the things!" have ever read "The Wealth of Nations". Most probably can't do it since centuries later the text is quite stilted.

There is no implied malice, that would require understanding what right and wrong are. It doesn't. The training so far for the various AIs seems to be whatever we can grab from the internet or scan from whatever books we can get our hands on.

Are you claiming LLMs are written to create fake social media accounts and launch a smear campaign against a project maintainer who refuses to apply the AI's PR? Because that actually happened. It was prompted to make a few requested changes to a program and get a PR accepted with those changes. And that's what the bot did, BY ANY MEANS NECESSARY.

Put another way, would you run a makefile containing several billion rules that you are not allowed to examine first? Because that's what giving agency to LLMs is. Or might you prefer a debug mode where it shows what actions would be taken, but cannot take them? That is what I mean by not giving it agency.

Issues with the big red switch cannot be addressed in software. But you knew that, right?

AI is quite different. Nothing in Kali is going to do anything really surprising. If you ask it to launch X,Y, and Z attacks against a.com, that's what will happen. It will not suddenly blackmail Joe Random nor dox the Postmaster General. It won't try to recruit a server at b.com to it's quest. It won't try to steal cycles from someone's heart monitor as a distraction. It won't go shopping for a professional arsonist on the dark web. It won't even submit a bogus brief to a court citing non-existent case law.

Note that of the absurd things I suggested, only one of them is complete fiction on my part.

MS has been doing that for a good while, which is why in the few cases I have installed Windows on something, I have had to boot a Linux rescue disk to download a network driver Windows was missing.

Way to over-interpret the example! It doesn't matter if it's client, server, embedded, isolated standalone. A method where root can "gain" root access is not a security issue.

Sorry, this is not the strawman you're looking for. I never commented on Mythos combing through Firefox.

Closed source has those vulnerabilities too, but only the bad guys know about them because the code is not open for public audit.

Many CVE's are somewhat junk of the form if you already have 3 ways to legitimately become root, you can use this 4th way too. OR (samn close to) OMG!!! Denial of Service attack, you can turn off the power and nobody can access the server!!! RED ALERT!!!

Others are real and need to be addressed, but the junk does more harm than good by distracting from real issues.

The real take-away for all of this is that AIs have no concept of cheating nor of right and wrong. They don't reliably honor even explicit instructions to not do something.

Give them agency and there will be havoc. When you tell an AI with agency to do something, mentally add the phrase "at all costs" to the end because the agent will. Still wanna try it?

But please don't ask it to bring world peace, because grave yards are quite peaceful.

Not only the sensors. The Insulet Omnipod Insulin pump is also a 24x7 device (I've done wear studies for it). Bath, swim, it keeps on pumping away.

Your timetable is way too aggressive. Yes, plant life will eventually die from lack of CO2 - but that's expected to be no earlier than around 400 million years from now - and possibly as far as 1 billion.

Even a stopped clock is right twice a day.