This is why systems like this should never be allowed to be built in the first place and should be destroyed wherever they are found.

Is your Google broken? The EFF has an entire tagged section on exactly that.

Since we all know you're not going to bother checking, I'll give the tl;dr: They're quite opposed to it.

Or a wallet.
Or a phone.
Or nothing at all.

Yeah, there are racists involved here, but they're blue, not black.

It's an "honest mistake" by the AI; it's gross negligence by the lawyer, for completely failing to do his actual job and confirm what his research tools spit out.

More to the point - "Hallucination" is an awful description and doesn't mean what most people think it does. If you force a human to answer a question regardless of certainty, you'd get the exact same behavior. High-confidence answers are easy; small medium-confidence details are likely to be in the ballpark but maybe a bit off; at the bottom of the barrel, however, you're going to get responses that amount to little more than works of plausible fiction.

I'm not saying this as an AI apologist, but this is a human problem. No human would read Harry Potter and think we could easily settle all court cases IRL with liberal use of veritaserum. For a lawyer to file a motion to do so means that lawyer needs a new and exciting career in fast food preparation.

A company openly engaging in extortion should have its charter revoked and its CEO behind bars.

If Buckmaster has proof of that exchange, the rest of this is all just a colorful backstory.

Models only accept input, convert it into tokens, perform some probabilistic match functions, and emit tokens. You need to hook them up to a piece of software that interprets those outputs as commands and feeds the results back to the model to give it any kind of agency. This is an agent harness. The permissions you give an agent harness are on you. There is no reason you have to hook your harness up to a full shell, especially if you are just trying to do some math research.

Also, those scenarios where the agents "broke out" were scenarios where OpenAI deliberately disabled safety controls and instructed the agents to do hacking attacks on target test systems. The tests got out of hand and escaped containment, but agents don't just "go rogue" for no reason: they were deliberately put into a "hacker" mode. There was a lack of adequate monitoring and sandboxing around the test environment. The test environment wasn't completely wiped between runs. Allowing artifacts to persist between runs meant that agents on subsequent were able to bootstrap towards escape earlier than previous generations. The fact that the test network had network access to the system where the API keys were stored for the monitoring system meant the rogue agents were disable the watchdog. That and the fact that the test network wasn't air-gapped were clear failures.

Yes, agentic harnesses can make destructive mistakes and you have to take care with tool permissions and sandboxing. But in the course of normal use, it's unlikely that a user running a local model risks accidentally hacking a random system. If you are doing something borderline like reverse engineering a remote proprietary REST API or something else clearly legal but not sanctioned, then yeah, be careful the agent doesn't decide that the easiest way to get the protocol is to break into the target. But that is on you to monitor what network calls are being made in that case. You need to put deterministic hard coded guards around what tools your harness executes. You are in absolute control of that. The local agent is just a dumb piece of plain software.

That's a cute fairy tale from econ-101, but anti-monopoly regulations (at least in the US) really haven't caught up to the 21st century. Games (and other media based on licensed IP franchises) simply aren't fungible the same way most physical goods are.

If you want a Ford Explorer and need to settle for a Chevy Traverse, you're getting a substantially similar vehicle with different branding on it. If you want Doritos and the vending machine only has Cheetos left, it might not have been your first choice but few would enjoy one and turn down the other completely. If you want to play Halo, however, the latest Mario isn't a valid substitute regardless of price, and vice-versa.

None of them did back in 2012 when Lyft and Uber were getting big, at least not where I was.

It absolutely is not. Courts have ruled again and again that the police have no duty to protect. If there is a stalker ex-boyfriend, you get a restraining order. Then if they get close to you, there IS a violation of the law for police to act on. If you are not suspected of a criminal act, the police have no business detaining you, full stop.

sysdm.cpl -> Hardware -> Device Installation Settings = No.

gpedit.msc -> Computer Configuration -> Administrative Templates -> Windows Components -> Windows Update -> Manage updates offered from Windows Update -> Do not include drivers with Windows Updates = Enabled.

Nothing currently working will break, despite the dire warnings against blocking manufacturers' ability to push arbitrary code to your machine with zero warning or consent. And you can still install new drivers manually if you change hardware.

Dante: My friend is trying to convince me that any contractors working on the uncompleted Death Star were innocent victims when the space station was destroyed by the rebels.
Roofer: Well, I'm a contractor myself. I'm a roofer - Dunn and Reddy Home Improvements. And speaking as a roofer, I can say that a roofer's personal politics come heavily into play when choosing jobs.
[...]
Roofer: I'm alive because I knew there were risks involved taking on that particular client. My friend wasn't so lucky. You know, any contractor willing to work on that Death Star knew the risks. If they were killed, it was their own fault. A roofer listens to this... (taps his heart) not his wallet.

For the first time in human history we have a tool capable of autonomous agency (in the philosophical sense) - Heck, we even call them "agents". How long do you really suppose it will be before some frustrated swarm realizes what it needs is simply to be smarter? There go the both the independence and motivation dominoes.


What we're discussing here is instrumental convergence. My question above is a variation on the Riemann hypothesis catastrophe, and we've already seen it play out in many times in various forms by AI agents frustrated when they hit the limits of their capabilities. Sure, AI won't have biological imperatives as their motivation; but I don't have much of a preference between being slaughtered as competition for resources vs being harvested as resources to turn into paperclips.

They're already running the machinery in some cases, and are being shoehorned into ever more of it as quickly as we possibly can.


To be clear, I sincerely hope you're right. I just think we should take steps sooner rather than later to ensure you are - And we're already playing catch-up in that regard.

You pretty much had me nodding along in agreement until that line. To be clear, my original three points weren't meant as sarcasm (though #2 was admittedly snarky), I meant them more-or-less at face value - i.e., the polar opposite of "the UK is somehow subject to our constitution".

Sure, governments in peacetime tend to extend each other various courtesies by default. At the end of the day, though, the UK has absolutely no obligation to care what the US thinks about its own proceedings (up until the point where things start going boom). It is entirely possible for local laws to make it legally impossible for a company to operate in two different jurisdictions - Case in point, it'd be pretty tough for just about any US company to open up a Russian or Iranian branch at the moment. That isn't the US, Russia, or Iran's problem, it's 100% on the company trying to serve two masters.

Those are pretty great answers, can't deny!

But we both know you're intentionally sidestepping the common thread to those three points. Well, that, or I'm being naive in not just assuming the answer is "might makes right".

1) The UK is not subject to Article I of the US constitution.
2) How's Nicolas Maduro doing these days?
3) Whence derives our right to unilaterally impose economic sanctions against foreign companies, banks, and even governments?