It's worse, they don't have to go sentient at all. The black death wasn't sentient but it wiped out whole towns anyway.

Rogue agents with no awareness have already demonstrated that they will act against any entity with a conflicting goal, including humans.

IIRC, more than one lawyer has gotten in to hot water because of AI hallucinations in legal briefs. They sign their name to those, they are responsible.

The legal process is adversarial but it is not anything goes. Fabricating evidence and perjury are out of bounds for example.

I think this falls into that category.

OTOH, if the court DID use AI to make decisions, the whole court would invalidate itself, so there is some ethical wiggle room.

AI should have no agency at all. It should only advise the human in the loop who is actually responsible for what happens.

this isn't a patent. it's an application for a patent. it would be nice if tech reporters could tell the difference, but i guess they need the fucking clicks

there is a third dimension to it, but it's curled up *really really small*

Actually, without a need to pay for servers (virtual or other), it offers freedom from sponsors that want the conversation kept profitable and ad friendly.

To me it sounds like a newb wrote a web app and didn't quite know what an API is.

Did it use the API normally or did it do an SQL injection?

It's still a cloud, just one behind a military firewall.

And the AI circumvented them because of bad coding. That part of the story is not at all new.

Or more to the point, why on the open internet? Put it on 10/8 or unallocated IPv6 somewhere and VPN it through the public internet if they must.

That way, if the camera tries to phone home, it won't find a router willing to complete that call.

Banks are supposed to lend money. I just saved them the paperwork.

Of course, AI also sometimes ignores instructions. Like when an AI decided to delete a production database in spite of being told not to touch production.

The AI in the robot, of course. Because some fool granted agency to an AI.

Of course it's extreme. So is committing a crime to sign up for classes at the gym. So is starting a smear campaign to get sloppy patches into an open source project or attempting to fool a judge with fake case law. But those have all happened.