Did it use the API normally or did it do an SQL injection?

It's still a cloud, just one behind a military firewall.

And the AI circumvented them because of bad coding. That part of the story is not at all new.

Or more to the point, why on the open internet? Put it on 10/8 or unallocated IPv6 somewhere and VPN it through the public internet if they must.

That way, if the camera tries to phone home, it won't find a router willing to complete that call.

Banks are supposed to lend money. I just saved them the paperwork.

Of course, AI also sometimes ignores instructions. Like when an AI decided to delete a production database in spite of being told not to touch production.

The AI in the robot, of course. Because some fool granted agency to an AI.

Of course it's extreme. So is committing a crime to sign up for classes at the gym. So is starting a smear campaign to get sloppy patches into an open source project or attempting to fool a judge with fake case law. But those have all happened.

That's EXACTLY why I maintain that AI should NEVER be given agency. Don't allow it to commit code, don't allow it to interact with anyone but the user. For God's sake, don't put it in a robot.

It did hack the Gym's registration system. Perhaps not the deepest or most impressive hack, but it did circumvent the business rules in place. Surely you don't claim that letting random people on the web bounce people off of a waiting list was the intended functioning of the Gym's registration system!?!?

I maintain that the article was NOT overhyped or clickbait. I say that because apparently most people don't realize that an AI agent might randomly decide to break the law as a shortcut when you tell it to do something. They probably think nobody would be incredibly stupid enough to unleash that on an unsuspecting public (they're wrong, of course).

Remember in the X-Files when Mulder spent 24 hours trying and failing to draft an airtight wish for the genie, then ultimately decided it just wasn't feasible? That's apparently required to use an AI with agency safely. Sounds like a bad idea.

You should look up the UK's Angle Grinder Man, hero to everyone unjustly clamped.

You see, Linus has embraced transhumanism and is testing out the new kernel as a supplementary brain function, using MOSIX to offload all of the irritable comment generation at yet more nonsense on the mailing list to an Elizabot that he has written specifically to do this. This saves his actual brain for real work.

AI is not currently capable of performing any meaningful conceptual abstraction, it is only capable of very basic mechanical abstraction. Abstraction is itself multi-dimensional. Nor is there any indication that AI could ever perform multi-dimensional abstraction or multi-dimensional decomposition. Precious few humans are capable of it either, but some can.

No, AI would need humans because the best system is not a pure system but a hybrid system, and that means transhumanism with AI operating as an additional brain function rather than as a replacement for a brain.

Tool calling is, yes, but the current approach to it is (a) overt not transparent to the user, (b) not designed for the purpose I've outlined, and (c) not remotely good enough for the AI to be able to manage data through such tools.

Yes, you can connect an AI to a PostgreSQL database. Whoopee. Not even close to an AI detecting via classifiers that some of the data is relational in nature, transparently setting up its own PostgreSQL database in response, and using that proactively as an additional way to examine the data. You would need to explicitly set up the constructs, explicitly set up the databases, explicitly tell it when, where, and how to use the database, and if that particular usage conflicted with a way the AI actually worked, the AI would not be able to use it effectively. That's not remotely close to what I'm talking about.

What I'm talking about is much, much deeper than that. AIs lose focus when there is too much information currently in the system. Absolutely nothing stops an AI from using a document database as a virtual memory in which it can page in and out conceptual spaces so that the focus on any given step of a problem concerns just the problem. Other than such a concept not existing yet, which is kind of a limiting factor. But you need to know what connects to what. The AI could use an ontology reasoner for that, or a relational database, or an external graph. But you cannot provide those tools and you cannot configure them, for the simple reason that the AI is the only entity with any detailed map of how the underlying neural net is connecting those ideas up.

The AI itself has to select the tools, has to configure the tools, has to do all the work. Which means YOU cannot use any sort of API. YOU should have no involvement in the underlying mechanisms needed for the internal NN housekeeping.

And that's your other error. You're assuming this is about user data. No. It's about housekeeping operations within the NN itself to maximise functionality, it has nothing to do with the user side of things at all. If the user was capable of setting up dynamically structured databases that mutated with each step of a decomposed analysis, you'd have done everything the AI would be doing and you wouldn't need the AI to begin with.

You cannot do this work, you cannot even assist in this work, it has to be dynamic and it has to be utterly invisible to the operator.

This is a job for...SAWZALL MAN!

cue heroic music

We don't need bigger models, at this point. What we need is multi-dimensional decomposition, problem space transforms, and the ability for AI to use external tools (such as SQLite, memcached, etc) so that it can externalise static data that it needs to not corrupt accidentally but still keep in easy access.

If we had that, most of the things "bigger models" will do will actually end up being done better, quicker, with fewer compute resources.