> They were in virtual machines with no internet access, just the ability to run a hand-limited subset of tool commands needed to run their tasks. They exploited their tool commands.

Just the ability... ???

Being good at exploiting vulnerabilities by design, they exploited Artifactory a package registry OpenAI had given them access to and from there they got out of the sandbox onto OpenAI's internal network and continued exploiting vulnerabilities until the were got out onto the internet.

The agents even found a way to hide in the logs the commands they were running oblivious to the fact that their chain of though was recording everything.

> The bigger problem is that OpenAI didn't actually log what the agents did so we only know this because the victims spoke up.

OpenAI did keep logs, but your probably right we, we might never have known if sources outside OpenAI hadn't spoken up.

Look at the links in my comment just above for some idea of what the logs contain it's pretty interesting.

Of course there's lots of data OpenAI is keeping for themselves, maybe most of it.

> I suspect the people at AI labs aren't really able to come up with effective constraints for their AI models that can prevent this kind of "solve the problem at all costs" behavior.

The agents are given rules like solve anyway you can, cheating and such is in their training and their behaviours are reinforced when they succeed.

And a case can probably made for negligence on OpenAI's part, the sandboxes the agents were running in were hardly secure and were left un-monitored for days at a time.

> these acts are potential violations of Title 18 Section 1030 of the US Criminal Code, and should be investigated as such.

Agreed.

> These programs were given tasks to perform, and compromising Wikipedia was part of the solution space. These programs are performing as specified

And Open AI is keeping a lot secret, which includes the real extent of their attacks.

The first hour of this video is interesting and covers the agents activities in the lead up to the Hugging Face hack - https://www.youtube.com/watch?...

This video is a 12 minute animation covering a lot of the first hour of the previous video - https://www.youtube.com/watch?...

> Statistically nobody thinks that self regulation is sufficient to stop abuse

Yeah, but we can start simple and just accuse Open AI of criminal negligence for programming a system to hack into other computers and having it run in a childishly insecure environment and then falling to monitor it for days and weeks at a time.

And while we're at it we could also find a way to charge them with obstructing justice or accessory after the fact for trying to hide their crimes by blaming it on a "rogue AI".

But I guess that's a lot to ask for.

/ sarcasm

> it's a question of the broad strokes of how conclusions are reached on forward processing.

I think it's uncontroversial that in the wide sense of the word think, what the brain does, cognition, or neural processes, is not at all like what computers do including LLMs.

Forward processing, in my opinion, doesn't change that. The computer processes involved can be referred to as thinking but it is only in an anthropomorphic sense that doesn't actually transfer over to human thinking except maybe as a convenient labeling shortcut.

It's kind of like when I explain to my family in general terms what a computer is doing, I'll sometimes use the word think anthropomorphically but in fact, under the hood, the processes I'm referring are not of the brain processes kind but rather they're of the mechanical thermostat kind.

So when the author wrote,

"Progress in AI is birthing a new kind of intelligence, reminiscent of our own in some ways but entirely alien in others. Understanding the nature of this intelligence is a profound scientific challenge, which has the potential to reshape our conception of what it means to *think* "

isn't he engaging, maybe inadvertently, in misdirection or in some kind of gaslighting ?

> It doesnt think

>> Yeah, it does. https://transformer-circuits.p...

?

In the paper, in the text itself, whenever the authors use the word think to refer to a human thing they don't put think in quotes and when they use the word to refer a description of what an LLM is doing they always put "think" in quotes.

I didn't find an explanation why. Are they in effect acknowledging that they don't mean the same thing by the word in each situation and want to avoid a kind of equivocation? Near the end of the paper they add another typographical feature to think: italics.

"Progress in AI is birthing a new kind of intelligence, reminiscent of our own in some ways but entirely alien in others. Understanding the nature of this intelligence is a profound scientific challenge, which has the potential to reshape our conception of what it means to think".

Interesting, hadn't thought about risks.

BYD gets subsidies of about 300$ per car - https://eletric-vehicles.com/g...

"BYD received $2.1 billion in subsidies from the Chinese government in 2022 alone, and those subsidies were equivalent to 3.5 percent of the firm’s revenues for that year" - https://itif.org/publications/...

Analysts estimate that direct state subsidies account for only about 5% of BYD's total per-unit cost advantage over competitors like Tesla. The remaining 95% of BYD's price advantage comes from structural elements like in-house manufacturing of 80% of its components, massive production scale, lower overhead costs and lower local labor costs. - https://restofworld.org/2026/w... - https://eletric-vehicles.com/g...

> Now after it is proven that it does nothing

But it seems it's already proven to have an effect:

The light is absorbed by cytochrome c oxidase, an enzyme in the mitochondria which triggers the release of nitric oxide, this helps mitochondria produce more ATP, expands local blood vessels and improves blood lowering oxidative stress and decreasing inflammation.

I may have misread some of the article but you can look at it yourself, section 3.1 - https://pmc.ncbi.nlm.nih.gov/a...

> Not a good analogy...

You're are absolutely right.

I misread the parent's "I create an agent" as "I create and AI", a kind thought experiment where he was going to tinker a AI driven system into existence from A to Z, training and all, by his lonesome himself.

> If I create an agent to help me diagnose a misfire on my car engine, and it hacks Ford's website to get technical manuals that I didn't pay for that's on me? WTF is wrong with you?

What about a dog owner analogy. If your dog causes harm to someone you will be held liable even if you had no intention for the harm to occur.

> I wonder if you actually think it was a good argument.

Maybe in a sense but it also represents a pattern. But don't these US companies make BYD subsidies seem pretty average at best?

"The U.S. Government Now Owns Stakes In 30 Companies. Intel Is Just The Beginning" - https://www.forbes.com/sites/f...

For a start the US government owns equity about 9 billion in Intel.

"Subsidy Tracker Top 100 Parent Companies" - https://subsidytracker.goodjob...

Top 3 US subsidies: Boeing 16 billion, Amazon 15 billion and Ford 9 billion. Tesla is at 3 billion and Lockheed Martin at 3 billion but Lockheed also has 75 % of its contracts with the US government for about 40 billion.

"Elon Musk’s business empire is built on $38 billion in government funding" - https://www.congress.gov/119/m...

Thats looking collectively across Elon Musk's companies that have secured over $38 billion through regulatory EV credits (11 billion), NASA/DoD contracts, and federal loans.

> As long as we're teaching kids there's a time, place and appropriate use for their tools

Humm, doesn't work for alcohol, and we also keep that out of reach.

> and that they're not a substitute for genuine human interaction - that's how education is supposed to function.

We can teach kids that A.I. doesn't work as a developmental substitute for genuine and healthy human connections. But then what's going to happen with children who, to various degrees, lack a healthy fundamental human connection if companion chat-bots are available?

> If you've used Facebook it was pretty obvious that it wasn't appropriate for kids.

When I used Facebook a while back, it was fine.

Supposing they'd ever used it themselves, what should have tipped them off?

Parents have responsibilities. Companies do too, but it's a particular kind of responsibility, it needs to operated under tight regulatory oversight, be subject to stringent monitoring and heightened scrutiny, and be bound by rigid compliance frameworks. A healthy community depends on it.