LLMs have certainly made people get really into conspiracy theories. *eyeroll*

The LLMs didn't just off on their own decide to look for hypervisor bugs. Users told it to look for hypervisor bugs. The timing means that multiple users did so at roughly the same time - probably in response to some news of some sort.

I'm not sure any of those have money at that level. Aaronson in the past when he was doing quantum computing work refused to take payment or stock from quantum computing companies, so I'd be shocked if he has any money. As for monetary concerns, Hinton explicitly quit his well-paying job at Google so he would be free to talk about AI risk. So the narrative here is the exact opposite. And when the companies talked about deliberately slowing AI growth, stock in AI related companies went down, which undermines the idea that this is anything about their investments. But there's also a serious problem here if someone like Coxon can be dismissed as not having enough knowledge or experience, but people who have been involved in the industry for a long time can be dismissed because they may have financial ties. So can absolutely anyone be dismissed as irrelevant, either lacking in expertise or being too tied in to the industry?

I'm not sure what an hour memory here means, since context windows are measured in tokens. Even six months ago, LLMs could think for well over an hour before producing an output.

This seems odd at multiple levels. First, the people in the foundation labs if anything have more of an idea what is happening than the general population. They are seeing how models are made, what they contribute, and how they go run off the rails. I'm also puzzled by the claim that we're not at the exponential. Two years ago, AI systems could barely do AMC and AIMIE problems, basic high school math contests. Then the AI got to be good enough to do well on Putnam results. Then, early this year, AI got to be good enough to solve multiple open unsolved math problems, including Erdos 1196 https://www.erdosproblems.com/1196 , the Unit Distance Conjecture https://arxiv.org/abs/2605.20695 and many others, and the trends there are just continuing. That's one field, but it is one where the standards are most objective about what is happening. That certainly looks like exponential growth. I'm also puzzled by seeing the labs cannot get the AI to improve themselves is a good standard. From the perspective of a lot of people who are concerned, once that's happening, it is likely too late.

What you mean here is you agree with Yann. But the point is that lots of people who are as qualified as Yann, disagree. So if, as in your first comment, your primary problem with Coxon is lack of experinece, you should find this situation alarming. Worse, your complaint about Coxon was lack of lab experience, but in your new comment you object to people being too deep in the labs to be objective. This leads to the weird situation where no one is in a position to be relevant to listen to.

It is possible that LLM tech has its limits. But people were saying it was hitting its limit 4 years ago, and 3 years ago, and 2 years ago, and a year ago. 4 years ago, people who were saying LLMs were limited would not have predicted they'd be solving unsolved math problems. All those predictions turned out to be wrong. Why should saying it now be more likely? And what if you are wrong here?

I'm glad you recognize the confirmation bias, yet I would dispute that it is in any way vindicated.

AI did not order the interception, put the assets in the air, or even submit the report. It got a manifest wrong in a chat session (not clear even it was its own error or related to the input), and that is... actually well-within common human error, not some special AI thing.

In fact intelligence is wrong all the time, and at all scales, but to name a big one, was AI to blame for the confidence of the existence of WMDs that was the basis for the Iraq war? Maybe it would have *averted* that war by helping with better assessment.

The story is sensationalized even without the effort to scaremonger on AI. There was never *any* chance of "causing a war" with China by interdicting a merchant ship in a officially declared maritime blockade zone, or we wouldn't have done it, even for nuclear components. In fact we've *already* intercepted Chinese vessels there and obviously they don't like it but just as obviously they don't feel like paying the cost of war with a major power to blow off their steam about it. (That's frankly a quite insane view of international relations.)

And it's not even clear to me that anything wrong was done. They double checked the intelligence, found it was incorrect, and ultimately took no action. Isn't that... exactly what is supposed to happen? Deploying your interdiction parallel to the intelligence check is a reasonable way to ensure a successful timeline in case something does need to be done. It's easy to recall planes in a false positive, not so easy to teleport them where they need to be for a verified true positive.

Unfortunately popular journalism is far from truth seeking. The dramatic and dangerous sells more copy, so that's what gets written. If you're going to use that as the raw material of your worldview, then of course it will wind up anti-tech and anti-science. At the very least I suggest finding a more reputable source of journalism to feed your biases than this slop.

So if the concern is these need to come from someone from a company not run by Altman, then we already have that. The OP is about that. And we've had containment breaches reported by Anthropic also. We also had a smaller scale but similar problem reported by an independent security group running an open-weight Kimi. You don't need to trust Sam Altman or anyone associated with him here.

The laws in question should just straight up be the transitive property. Any paid agent of government should be restrained by all of the restraints on government, plain and simple.

Low pay? Median US salary is $51K. Median US police officer salary is $77K. There might be high burnout and turnover, but the pay is generally pretty good all things considered.

Be very careful with 4 and how it's worded. You'll end up with a lot of police officers shoving suspects up onto the hood of their car all the way to the windshield to block the camera, then arresting them for "obstructing" the camera.

I think you're forgetting that, if that search leads to the car being pulled over, then to a search of the car, and they find a dead body in the trunk, the way the concept of the fruit of the poison tree is supposed to work someone could get away with murder for that "haha", etc. The dead body in their trunk and all additional evidence gathered as a result of that too could potentially become inadmissible leaving the prosecution with an obvious murder, but no evidence. The police are supposed to take their work - including the paperwork part of it - seriously.

Yeah, I'm sure those Genovese cable car passengers loved their flyover (under) experience.

A flyover isn't the problem here. The statement is.

TIL.

Suicide at age 52. Shame. :(

If you want to dismiss Coxon feel free, but he isn't the only one in the article. And outside the article, we have a lot of people who are concerned who are hard to argue aren't experts. Geoffrey Hinton and Yoshua Bengio both got Turing Awards for their work in Deep Learning. (The third who got it is Yann LeCun who isn't worried at all, but Hinton and Bengio is 2/3.) Similarly, Scott Aaronson, who wasn't even that worried about this a few years ago is now highly concerned. Aaronson is one of the world's leading experts in computational complexity, who discovered the algebraization barrier among other things, and whose work lead to the AI watermarking systems. Aaronson just wrote this essay https://scottaaronson.blog/?p=10062. So sure, go dismiss Coxon, but it doesn't impact remotely all the others who are clear experts and are sounding an alarm here.

What needs to happen to decide these aren't PR campaigns and are actual warnings? What would need to occur? Knocking out a power grid? Hacking the Pentagon? Wrecking with air traffic control and making planes crash? I ask in part because when these incidents started happening two months ago, people said that they were just PR and that if they were real we'd start seeing more of them and that seems to be exactly where we are, and yet these are still getting labeled as PR campaigns.

The original quote they are riffing on is "The beatings will continue until morale improves." Essentially this is a variant of the "make a threat, but in a joking fashion, but really functionally a threat" sort of thing that dangerous and abusive people will do. Trump uses versions of this all the time, and it isn't surprising that Hegseth and his people have picked up on this. I'd call this another warning sign about fascism but given everything else that's going on, including a grossly illegal war in Iran, Trump functionally threatening to undermine the upcoming election, and Trump having removed generals and other higher-ups in the military that he sees as insufficiently loyal (read, more likely to actually follow the law and morality when given an illegal order), and this incident becomes almost part of the rounding error.