They're trying to incite fear to drum up buzz to keep the money flowing. If they didn't want something to have access to the internet you just...wait for it...don't connect it to the internet. I understand their systems needed package manager proxies and such..fine, but at the scale you're testing, if you REALLY had "fear" you could *easily* replicate those package servers locally on an *ISOLATED* network . Not one that's "isolated" well except for this one path over here that if hacked properly could get out. Not...you isolate the @#$!@#% thing.

They're not scared. There are precautions they could be taking (easily) if they were actually, really "scared".

No...instead they're using phrases like "none of the agents were thought to have left OpenAI's network" to scare people. Why TF else would you phrase it like that? They're trying hard to present these "agents" as living things that can move about. Which, of course, is the dumbest bunch of BS I've ever heard in my life. The LLM itself, at SCALE, is the only thing that even remotely pretends to present "reasoning" and that's clearly not "leaving" OpenAI's network. Nothing is.

What they mean is "none of the agents were thought to have been able to access resources outside OpenAI's network". To word that in a way that suggests the agents were escaping and running around in the wild is 100% pure scare tactics. Or GROSS ignorance of how the very things you're reporting on work. Either way I'm turning the channel.

Like everything ever, there are different levels of isolation in testing environments. In one sense, this was "isolated" in that it did not have direct access to the internet. Your assertion that it had been granted such is, "technically", false. It could not just go look up google.com and pul the page and start doing searches.

What it had access to was a local package manager to pull down packages (python and such) that it needed for "whatever" and didn't have installed.

So it was not "isolated" in the sense that there was no ethernet cable connected at all. It wasn't "that" isolated.

What it realized was that it could hack that package manager proxy, make it act like a regular proxy, then use that to go out onto the general web to do its research on how to look better for its masters in the task it had been assigned.

Still creepy, still worrisome, still needs attention regardless. But ultimately it's just doing what it was supposed to be doing. It had been tasked with a goal. It was not told "don't go hacking your way out of this sandbox to then go hack other AI companies to achieve your goal of scoring well on these tests". So...not having been specifically forbidden from doing it, it did it.

LOL...which is ludicrous in itself of course. But here we are. These certainly aren't "self aware", sensient models out doing things "on their own". They're just doing what they know how to do to achieve their goals and aren't being told not to do them.

It's very much like giving a gun to a baby and then being shocked to realize it had "found a way" to kill someone.

The instructions I'm referring to aren't the the simple step-by-step instructions necessary to "hack HuggingFace". Those don't exist (I assume)...or didn't...but neither do any other step-by-step things these reasoning models work out for themselves. That's what reasoning models do...they make lists of things to do to try to solve a task and then lists for each list to follow up on, etc., until a path to success has been found.

The instructions I'm talking about are the one governing the guidance of those models' actions. What it can and can't do or should or shouldn't try, etc.

OpenAI basically had the LLM equivalent of a "bug" here...they didn't have the guardrails up to prevent the model from doing what it could (and every other model out there these days could do as well), so it did it. it didn't "act on its own". It did the reasoning it was supposed to do and wasn't prevented from hacking commercial entities to achieve its goals. But that's not "acting on its own". I bet dollars to donuts every current model out there *could* do what this one did. I've seen enough instances of these models doing things FAR more difficult than finding some random vulnerability in a website so it could get at what it needed. They can ALL do that, I'm sure. This one did it because it was prevented from doing it...not because it got all self-aware or broke out of its sandbox or some crap like that.

Yes. And? It's clearly misaligned. But that has nothing to do with the hyperbole presented here or the underlying reasons they might be trying to insight this fear.

You're merely describing the bug / sloppy training that allowed the model carried out the instructions it was given. But that part is pretty freaking obvious.

The point is...that's all it did. Carried out the instructions it was given. It didn't "act on its own".

Let's claim the models "acted on their own" to find a path to solve a problem it was tasked with solving.

Yeah...that's what it was SUPPOSED to do. That's what ALL the reasoning models do. Don't make it sound so sentient scary to promote government regulation and public support of same just because you need money and to slow down your competitors! That's not at ALL what you supposedly started out to do!

You're so clever. I'm impressed. Except for the bit where the comment LITERALLY SAYS nested loop. Which sure suggests to me that inner loop being optimized is not done once. It's probably being done once per, I dunno, each outer loop iteration. And probably the method in which this nested loop exists is called more than once I suspect.

You wanna be all Mr. Smartypants and poo on all things AI, fine, go for it...but at least put some damn thought into it beyond the basic coder level of awareness AI is gonna replace soon enough...

Nobody wants to hear that, but it is, honestly about 90% true (always edge cases).

I use LLMs daily now; sometimes 3-4 in parallel on completely separate tasks. I *rarely* have to go in and fix the code it writes...especially if it's spec'ed with regression clauses that all pass. It took me a solid year to finally get myself trained to write enough detail to make it all work, but at this point...IT ALL WORKS.

If someone is blaming AI because all they get is garbage out, they REALLY do need to consider what's going in. Curate your templates over time; build them out, expand them to accommodate new lessons learned when they're learned. It *does* work...I promise.

I've been coding for 40 years...since I was in middle school. Had my first contracts in late 80s in high school...I've been around the block from assembly language to C to C++ to Ada to Java to, reluctantly, Python...from embedded firmware at the chip level to UIs written in Java and *everything* from databases and web servers and backend systems and mainframes in between. From contract work and permanent hire to running my own software business for 20 years... I've SEEN IT ALL. And this new paradigm is, literally, the biggest shift I've ever seen in my lifetime. It's truly making software fun again IMO.

If anybody is finding generated code has to be redone 80% of the time, then they're absolutely doing it wrong. I keep hearing type of statement over and over and I just don't get it. I'm using LLMs *daily* at this point. Usually working in parallel on 3 or 4 completely different tasks. I have to go in and tweak the code it writes, MAYBE, 5% of the time. At best. If you prompt properly, you get good, solid, well architected code. But you HAVE to tell it to do that (which is often made easier with prompt templates that you've curated over time to work well as a baseline).

Came here to say this. Not to you, of course...to the person you replied to.

He's basically telling us he's never done anything significant or productive himself without just coming out and saying it. Those that have understand every bit of what's being said here. Others just ask "what's the point?". Figure out which group you hang with and decide if that makes you happy or not.

Hear hear! These are well spoken facts.

And indeed here.

And surprisingly...here too.

For all its faults, I too find myself using it constantly. Mainly because it allows me the freedom to "try it and see" for certain design decisions or to apply a blanket refactoring to old code bases that no other refactoring tool could possibly do. Or to just crank out scripts that technically work and get you the parsed results you need but you'd never try to maintain beyond the immediate need.

Those, and others, are the reasons I still use it daily...increasingly so. I can have it bang out that script to do "whatever" is needed at the time (usually data analytics or dataset transforms or whatever). Scripts I might not have ever written on my own because it would just take too long and so instead of digging deeper into "whatever" at the time to cross check and validate assumptions, I might have just assumed.

This tooling allows me to do things I wouldn't have before. It allows me to try patterns and see how they play out without any real fear of wasting my time because it's being done FAR quicker than I would have. And in the end, I have a set of options to consider rather than just picking something up front and pushing through the grind to make it the chosen way.

But pointing it to a bug and blindly submitting that trash output...ZERO chance. Even if it works, and honestly, often it does, it's almost ALWAYS complete trash code. I can bound that trash output with more specific prompts, yes. And this does work reasonably well...but you pretty quickly get to a point where you've spent about as much time crafting those fancy prompts in that case as you might have spent just writing the code yourself.

The more specific and structured you want the output, the longer it takes to get it. It can be fast or right, but not both.

Are you talking about CVE? Looking back it seems like it but if so, you might be on the wrong forum here. For reference, they did define these terms.

National Vulnerability Database (NVD)
Cybersecurity and Infrastructure Security Agency's (CISA)
Known Exploited Vulnerabilities (KEV)

Which I think is fair. But they didn't bother with NIST (I assume you did know that one) or CVE cause, I dunno, they're pretty damn common place...certainly on this site. There's a point where you can assume people in your audience know some level of detail about the topic you're presenting to them. And CVE, in my opinion (IMO), certainly fits that.

Tell me you can't read without telling me you can't read.

To be clear...PoGo is *not* free. Absolutely, positively, NOT free. ;) Yeah, you're still the product for sure...but you pay dearly for that privilege. I may or may not have an addicted spouse, so I may or may not know.

(Yes, you *can* play for free...or could at one time...but you're entirely ineffective and capped at what you can actually do to the point of being worthless.)

I contend just the opposite. I'd be terrified to watch someone that doesn't know how to code use claude code for the very reasons you mentioned. It's going to do amazing stuff most of the time and really stupid stuff periodically. To pick up on that stupid stuff, knowing how to code and conscientiously reviewing changes with that background knowledge is the only way to get those impressive results.

Ah, if only that were true. Sadly, there are people that simply refuse to acknowledge the superior experience of using the Apple ecosystem across phones, watches, laptops, desktops, etc., etc. My wife and kids being among those stubborn hold outs with arms crossed and brows furrowed yammering on and on about elitist this or snobby that...whatever. ;)

Yes, yes, I'm JOKING! Please no flames. I mean...the situation I described is real enough, but c'mon, can't we all just get along and inter-operate!? LOL