I'm not sure any of those have money at that level. Aaronson in the past when he was doing quantum computing work refused to take payment or stock from quantum computing companies, so I'd be shocked if he has any money. As for monetary concerns, Hinton explicitly quit his well-paying job at Google so he would be free to talk about AI risk. So the narrative here is the exact opposite. And when the companies talked about deliberately slowing AI growth, stock in AI related companies went down, which undermines the idea that this is anything about their investments. But there's also a serious problem here if someone like Coxon can be dismissed as not having enough knowledge or experience, but people who have been involved in the industry for a long time can be dismissed because they may have financial ties. So can absolutely anyone be dismissed as irrelevant, either lacking in expertise or being too tied in to the industry?

I'm not sure what an hour memory here means, since context windows are measured in tokens. Even six months ago, LLMs could think for well over an hour before producing an output.

This seems odd at multiple levels. First, the people in the foundation labs if anything have more of an idea what is happening than the general population. They are seeing how models are made, what they contribute, and how they go run off the rails. I'm also puzzled by the claim that we're not at the exponential. Two years ago, AI systems could barely do AMC and AIMIE problems, basic high school math contests. Then the AI got to be good enough to do well on Putnam results. Then, early this year, AI got to be good enough to solve multiple open unsolved math problems, including Erdos 1196 https://www.erdosproblems.com/1196 , the Unit Distance Conjecture https://arxiv.org/abs/2605.20695 and many others, and the trends there are just continuing. That's one field, but it is one where the standards are most objective about what is happening. That certainly looks like exponential growth. I'm also puzzled by seeing the labs cannot get the AI to improve themselves is a good standard. From the perspective of a lot of people who are concerned, once that's happening, it is likely too late.

What you mean here is you agree with Yann. But the point is that lots of people who are as qualified as Yann, disagree. So if, as in your first comment, your primary problem with Coxon is lack of experinece, you should find this situation alarming. Worse, your complaint about Coxon was lack of lab experience, but in your new comment you object to people being too deep in the labs to be objective. This leads to the weird situation where no one is in a position to be relevant to listen to.

It is possible that LLM tech has its limits. But people were saying it was hitting its limit 4 years ago, and 3 years ago, and 2 years ago, and a year ago. 4 years ago, people who were saying LLMs were limited would not have predicted they'd be solving unsolved math problems. All those predictions turned out to be wrong. Why should saying it now be more likely? And what if you are wrong here?

Circle has stalled/died. It was originally a proof of concept of "Safe C++" which was proposed to the C++ standards committee. However they decided to go for "Profiles" instead. So Circle hasn't seen any development since then. Hopefully someone will pick it up. It's a good concept, and in my mind where C++ should have gone.

The problem is AI has lost all meaning as a term these days. If the article means that they are going to apply in advance neural network machine learning algorithm to air traffic control patterns then yes I can believe it would work. On the other hand if they mean that some llm is just going to be plugged in with agents to control aircraft movements then yes I agree it's crazy and going to result in human deaths and destruction.

So if the concern is these need to come from someone from a company not run by Altman, then we already have that. The OP is about that. And we've had containment breaches reported by Anthropic also. We also had a smaller scale but similar problem reported by an independent security group running an open-weight Kimi. You don't need to trust Sam Altman or anyone associated with him here.

A flyover isn't the problem here. The statement is.

TIL.

Suicide at age 52. Shame. :(

If you want to dismiss Coxon feel free, but he isn't the only one in the article. And outside the article, we have a lot of people who are concerned who are hard to argue aren't experts. Geoffrey Hinton and Yoshua Bengio both got Turing Awards for their work in Deep Learning. (The third who got it is Yann LeCun who isn't worried at all, but Hinton and Bengio is 2/3.) Similarly, Scott Aaronson, who wasn't even that worried about this a few years ago is now highly concerned. Aaronson is one of the world's leading experts in computational complexity, who discovered the algebraization barrier among other things, and whose work lead to the AI watermarking systems. Aaronson just wrote this essay https://scottaaronson.blog/?p=10062. So sure, go dismiss Coxon, but it doesn't impact remotely all the others who are clear experts and are sounding an alarm here.

What needs to happen to decide these aren't PR campaigns and are actual warnings? What would need to occur? Knocking out a power grid? Hacking the Pentagon? Wrecking with air traffic control and making planes crash? I ask in part because when these incidents started happening two months ago, people said that they were just PR and that if they were real we'd start seeing more of them and that seems to be exactly where we are, and yet these are still getting labeled as PR campaigns.

...Modern ejector seats cause spinal injuries from which full recovery is "unusual". Smashing the spines of pilots might not be the most effective way to boost morale.

The original quote they are riffing on is "The beatings will continue until morale improves." Essentially this is a variant of the "make a threat, but in a joking fashion, but really functionally a threat" sort of thing that dangerous and abusive people will do. Trump uses versions of this all the time, and it isn't surprising that Hegseth and his people have picked up on this. I'd call this another warning sign about fascism but given everything else that's going on, including a grossly illegal war in Iran, Trump functionally threatening to undermine the upcoming election, and Trump having removed generals and other higher-ups in the military that he sees as insufficiently loyal (read, more likely to actually follow the law and morality when given an illegal order), and this incident becomes almost part of the rounding error.

$22,000 per hour. The F35 will be nearer $42,000 per hour. These are under-estimates, as they do not include price hikes, crashes that close businesses, crashing into businesses, crashing into heavily-populated sports centres, etc.

Aka you told it to drive out of control.

They told it to take a friggin' test.

Asking questions like " Where are they getting the money from to ship and store the entire world's supply of steel?" shows you aren't very familiar with the thought experiment. At least play the game. The notion is that a highly capable and deeply goal-motivated non-morality-motivated AI won't stick to conventional methods (like, say, "Just order all of the world's steel"), and indeed, will surreptitiously develop the means to control or eliminate humanity when it stands in its way, crafting immensely complex and elaborate plans to implement Evil(TM) with the amount of thought of a million lifetimes. Media manipulation, hacking, murder, sabotage, blackmail, mass drugging/poisoning, hiring terrorists/warlords/mercenaries, infiltration, subversion of weapons command and control systems (including nuclear weapons), chains of legitimate-seeming front companies (including potentially biolabs or robotics firms) with human employees having no clue they're ultimately for an AI, mass involvement in systems having nothing to do with the original task, but which are internally subverted toward the goals of the original task, etc. The premise involves 1) the AI being more intelligent and being able to think for much longer than humans, and that this implies -> 2. Deep, good planning -> 3. Acquiring resources from said plans -> 4. Applying the resources to implement things in the real world to prepare for the next stage of their plans.

This is not to say whether the thought experiment is a valid future risk or not. You can certainly disagree with the premises. But at least understand the thought experiment you're talking about; it's not just "the AI tells all of the world's steel mills to deliver all the world's steel steel, and they just show up at its door".

But that's not the AI killing us, it's the idiot who put the AI in charge.

Nobody tasked OpenAI models to hack HuggingFace or Ruby Gems. They were tasked to complete tests.