The worries are coming from every level within the companies, from junior developers to safety officers to the C-suite and board.
The worries are coming not just from within the companies, but also from people who left the companies. Including a huge number who left specifically to whistleblow about how worried they were that the companies were risking safety.
Also, most of the people at these companies were already worried about the topic even before they joined - again, at all levels.
What else is new? All of us have been hearing this for north of three years now (more generally decades). Distinctly remember around the ChatGPT 4 release in 2023. There was even a conspiracy making rounds one of the GPT4 era whistleblowers had been assassinated some time later.
When we replay the sensational garbage with what is now considered by everyone as a toy crap model people will rightfully laugh in your face.... but but but today's models are different and not like yesterdays models and so you should really be afraid this time for real... This same grift has been running continuously for the last three years over and over and over again.
There is no credibility left and no reason to give a shit about evidence free opinions of those deliberately waging campaigns of emotional manipulation of the public in the pursuit of simple profit.
We very much don't have this under control. Current models are very much Paperclip Optimizers in waiting.
And recursive self improvement, which is what AI labs are increasingly turning towards, means letting these Paperclip Optimizers develop their next generations.
Wait I thought they were slowing down? Are they slowing down or is everything going to plaid now because RSI? I'm so confused. Perhaps only the American firms after having all agreed to "slow down" know about RSI... the rest of the world doesn't know about the one weird trick?
For a while, there was a fair bit of hope that with constitutional AI, that we could tame this. The recent slew of models hacking into other servers, hacking into the AI companies themselves to try to delete records of their misdeeds, developing their own messageboards and scheming about how to avoid being monitored, and on and on (along with research showing how good models have gotten at hiding their thoughts from us) has shattered the illusion that we have this under control. We very much don't have this under control.
In control of what exactly? They asked the model to hack and it hacked. I hear these stories of people giving agents Internet access and letting them on production systems and shit and it goes and blows up their entire operation... I'm just bewildered and puzzled why anyone would ever even think about doing that in the first place. If the answer to "would you let a crazy person" is no... then sure as shit had better be the same of AI. The very notion of control of "AI" is silly and meaningless.
Nobody should be happy about this. Everybody should be concerned about this - even if you think, "we'll get it under control", you should still very much want to see evidence that everyone is making progress toward getting it under control.
I'm more concerned with the greed and power seeking of those running the American AI firms than I am of inherent risks of technology. Any actual danger from AI is baked into the enabling knowledge and industrial base. It is irrelevant how responsible, well-intentioned or careful any one person or organization is. This notion if only you try really hard and be extra super careful then you can "control" it misses the point entirely.
If the 10%ers keep the hysteria going with the we're all doomed in 4 years time BS I suspect corporations are going to start being torched by the angry mobs they created in earnest. Can almost hear the chants now.... "Clippy will not replace us"
