I always figured that people who genuinely believed their employers were likely to kill literally everybody in the near future might burn down a bit more than that, but so far it's mostly been tweets

Having to poke through an about: page and find the thing to toggle to turn off this specific AI feature but not that one, and knowing they'd change it on you a couple of releases later, never filled me with confidence.

One switch in a readily discoverable location in the settings menu and it's all gone? Just the job, killed it now on the laptop and the mobile both.

This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.

That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.

I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.

And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.

The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.

They don't care because they don't believe that.

When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.

> approximately eight hundred billion ... oops, that's eight hundred million, of course. Ten percent probability, times the current population of the Earth. The expected consequences of allowing the AI industry to continue their work, according to people who are oddly calm about the prospect.

The actual post is behind a login wall, but I don't see from the BBC article that he's even advocating any action. Is he demanding the immediate shutdown of the data centres? Raids on the labs? Coordinated global action to impound and destroy the equipment? Life imprisonment for anybody found training large neural networks? Rogue researchers declared hostis humani generis? There's no mention of it.

That's what I'd be advocating at the very least. Then the question of whether to take direct action personally on top of that would be one of practicality - can I lay my hands on the right equipment, how can I do the most effective damage, do I have access to high value targets, will it be dramatic enough to frighten new recruits away from joining the industry, will others be inspired to follow my example, should I take the time first to form a team for more extensive action, that kind of thing. It wouldn't be a question of whether it should happen, only one of whether those are the right tactics. Yes, in all probability I'd decide I wasn't personally cut out for terrorism or assassination, but not everybody would decide so. It would be a reasonable choice in the circumstances.

If the expected bodycount from the operation of this industry over the next ten years was approximately eight hundred billion, then that would make AI researchers the single most dangerous group of human beings that has ever existed. The National Socialist German Workers' Party, previous title holders for the worst people ever, probably didn't kill even a tenth as many as this man says he expects AI firms to kill in roughly the same timescale. That is the comparison that would frame my thinking about AI researchers and what we should do with them, if I believed what he says he believes.

And I just don't see that kind of thing coming up in these conversations. People calmly and academically discuss their probability estimates as if it's a question of pure theory, not the imminent destruction of all humanity. If they really believed what they say, there would certainly by now be AI Luigis everywhere, data centres would be burning, and people would be quitting the industry all over the world for their own safety. That simply isn't happening. So I don't believe they actually do think that this outcome is likely. Actions speak louder than words, and their actions say they don't believe a word of it.

... then I would be demanding that every single person involved in the AI industry be rounded up and shot. All of them. My former boss, my coworkers, our competitors, all of them. I would be advocating a world in which is was worse to be found with LLM weights on your computer than to be caught with the worst possible kind of CSAM.

But apparently the idea that his former employer might literally kill literally everybody in just ten years' time isn't that big a deal for this guy. Go into the office one day and take a Second Amendment approach to as many of the frontier research team as possible? No... let's just whine about it on the internet!

Which is why I'm not at all convinced he believes this. If he believed that, this action is too small a response. Nothing short of Butlerian jihadism is an appropriate response, if you actually believe this.

If you were worried about being chased by kill bots, go and watch some videos from drone operators in Ukraine. Robots don't need feet for that job.

I think you CAN have it both ways: Announce at the beginning: "If you want, I will give you a B- (or whatever grade works) but to get this grade you must promise never to set foot in the class and never to speak about the topic of this course with anyone on campus. Otherwise, we will explore the syllabus completely free from the stench of grade-grubbers who are at the school in order to get some sort of imagined advantage in the world." Then, no tests. Grades or evaluations will be felt out as the class goes. The B- people will get the bullshit degree that they paid for, they might well succeed in the dumb-ass world, who gives a shit.

This crisis hasn't been brewing: university has always had two conflicting purposes: investigate interesting stuff deeply with people who have devoted their lives to this; and a mechanism to reproduce and reinforce the class system. This second purpose hides and piggybacks on the first.

LLMs are a new and developing set of techniques and computer people should be experimenting. The "speed" of development has been turned into "speed" of trying to BLOCK experimentation. We are being instructed to think of "AI" as something that only a few behemoth companies do. Like how "computers" are just MS and Apple, or "phones" are just AT&T, or "Christians" are just in Rome, or "voters" are only landowners, "writers" are only monks... This article is part of a bald and disgusting POWER-GRAB.

US companies like OpenAI and Anthropic WANT regulation in order to block weaker players from developing competing AIs because they can't conform to the regulations. The "costs" of running AI will be the cost of having massive data centers with all kinds of anti-copyright regulations, and a proliferating list of other phony safeguards, including "safe guarding" the companies' own interests in parallel with being moral police, fear mongers, and buck-passers. Chinese and independent developers will train on pirated data and will not need data centers because they will be able to run locally. This will give them some competitive advantages - so the US companies will push to make training data, i.e, all writing and reading, be regulated – no one will be allowed to read websites unless they are logged in and ID authenticated. US AI companies will pay for access. Non-payers will be accused of violating international trade pacts, etc. Their AI models will be banned and there will be a movement to consider them criminal, selfish, etc. The main foreign policy of the US will be to get other countries to agree to license training data, and this issue will dominate US policy. When I (me, a programmer) want to program/train my own AI, it will be made illegal or at least very difficult. Right now it is difficult from a tech resource and org resource POV. But soon all the difficulty will be strictly from a regulatory POV. My access will increasingly need to be monitored and blocked.

But all of this is just a continuation of the DRM fights and the Free Software battles of the last decades. It isn't even a huge shift. It is a disgusting and short-sighted vision of our values. The worst parts of our culture and business practices have come to dominate the world.

... sadly for the Americans, the rest of the world now knows they can't count on a US based provider for this kind of thing any more.

It was uncomfortable enough relying so heavily on American software back when it couldn't be switched off remotely on the say so of an idiot. Today it's an intolerable risk.

I tried hard in the last 2 months: I bought Claude, came up with a project I thought seemed reasonable: JS-based rich-text editor with plugin system. Existing solutions (CKEditor, TinyMCE, Quill, etc etc) are old, unwieldy, sometimes proprietary, and modern browsers have many newly supported features... My goal was an HTML web component of /

I tried to carefully prompt. Before coding I used Claude to help research the issues involved (dead standards, browsers handling edges cases differently, generated HTML questions, etc.). Claude was thoughtful and reassuring. I knew it would be more complex than Claude kept insisting, but that's OK. As usual with LLMs, at first I was more than impressed, I was blown away.

Still, bugs. That's expected. Fixes were easy and it was amazing how Claude understood the issues. But the more I tested, the more the bugs proliferated. Some issues activated Claude to rewrite whole architectural parts of the codebase, which broke dependencies. Fixing the dependent stuff introduced new bugs. I slowly had to learn more and more about the implementation specifics. More and more I had to audit the code changes, revert, try again. Soon I found myself questioning Claude's approaches in what seemed to be subtle ways. At times I was forced to really dig in, and the code – which looked so clean and organized – was a true spaghetti mess. Out-of-date comments. Repeated blocks of functionality with small differences. Convoluted back-and-forth paths across files, functions, classes. Each plugin had drifted to requiring its own long list of specialized one-off supporting worlds of code. Basic browser functions got overwritten with convoluted bespoke mish-mash slop with long interruptions of exceptions work-arounds and crazy shit.

Maybe the thing works. But the bugs are brutal! Everything is delicate! I've lost track of what the hell is going on.

But all of this was very familiar! It all looked like what USED TO happen to me before I got experience. What happened when I instructed programming newbies to take a crack without supervision. What happened when someone paid $5 to Upwork for something the boss thought would be easy.

We are not there yet. Not even close. It is 1998 and we are using for layout with the "100% td width" work-around.

well once they crossed 2 billion the signed integer representation made losses into profits