This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.

That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.

I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.

And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.

The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.

They don't care because they don't believe that.

When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.

> approximately eight hundred billion ... oops, that's eight hundred million, of course. Ten percent probability, times the current population of the Earth. The expected consequences of allowing the AI industry to continue their work, according to people who are oddly calm about the prospect.

The actual post is behind a login wall, but I don't see from the BBC article that he's even advocating any action. Is he demanding the immediate shutdown of the data centres? Raids on the labs? Coordinated global action to impound and destroy the equipment? Life imprisonment for anybody found training large neural networks? Rogue researchers declared hostis humani generis? There's no mention of it.

That's what I'd be advocating at the very least. Then the question of whether to take direct action personally on top of that would be one of practicality - can I lay my hands on the right equipment, how can I do the most effective damage, do I have access to high value targets, will it be dramatic enough to frighten new recruits away from joining the industry, will others be inspired to follow my example, should I take the time first to form a team for more extensive action, that kind of thing. It wouldn't be a question of whether it should happen, only one of whether those are the right tactics. Yes, in all probability I'd decide I wasn't personally cut out for terrorism or assassination, but not everybody would decide so. It would be a reasonable choice in the circumstances.

If the expected bodycount from the operation of this industry over the next ten years was approximately eight hundred billion, then that would make AI researchers the single most dangerous group of human beings that has ever existed. The National Socialist German Workers' Party, previous title holders for the worst people ever, probably didn't kill even a tenth as many as this man says he expects AI firms to kill in roughly the same timescale. That is the comparison that would frame my thinking about AI researchers and what we should do with them, if I believed what he says he believes.

And I just don't see that kind of thing coming up in these conversations. People calmly and academically discuss their probability estimates as if it's a question of pure theory, not the imminent destruction of all humanity. If they really believed what they say, there would certainly by now be AI Luigis everywhere, data centres would be burning, and people would be quitting the industry all over the world for their own safety. That simply isn't happening. So I don't believe they actually do think that this outcome is likely. Actions speak louder than words, and their actions say they don't believe a word of it.

... then I would be demanding that every single person involved in the AI industry be rounded up and shot. All of them. My former boss, my coworkers, our competitors, all of them. I would be advocating a world in which is was worse to be found with LLM weights on your computer than to be caught with the worst possible kind of CSAM.

But apparently the idea that his former employer might literally kill literally everybody in just ten years' time isn't that big a deal for this guy. Go into the office one day and take a Second Amendment approach to as many of the frontier research team as possible? No... let's just whine about it on the internet!

Which is why I'm not at all convinced he believes this. If he believed that, this action is too small a response. Nothing short of Butlerian jihadism is an appropriate response, if you actually believe this.

If you were worried about being chased by kill bots, go and watch some videos from drone operators in Ukraine. Robots don't need feet for that job.

... sadly for the Americans, the rest of the world now knows they can't count on a US based provider for this kind of thing any more.

It was uncomfortable enough relying so heavily on American software back when it couldn't be switched off remotely on the say so of an idiot. Today it's an intolerable risk.

...a lot of people online were saying there was an upcoming event called The Halvening that would change everything.

I guess this is it?

Seems to me 'dead' for a taxi isn't 'dead' for a static power bank. If I'm running a taxi I've got hard limits on how large my battery can be and how heavy, and I want to maximise the mileage I get between charges, because while my taxi is charging it's not out on the road earning money. When that battery is keeping only maybe 80% of its original design charge, and now I have to schedule one recharge too many per working day? Bang goes my business plan, so I'm replacing it.

If I'm storing energy for the grid I'm a lot less worried about that. It only stores 80% of what it did when new? Better than nothing, and the taxi firm is selling them off cheap. I'll stack them up!

Perhaps not, but if you pick your moment right then permanently stopping the work of some of the most talented researchers there could very well make a difference. A spectacular incident that makes the headlines might also deter others - bright graduates might decide it's far safer to take up a different line of work, subcontractors and suppliers might decide doing business with AI firms isn't worth the danger, investors might figure the increased risk of loss of premises and equipment into their projections, that kind of thing.

If people genuinely believed AI takeover was a real, present and imminent threat, then they wouldn't just be publishing essays online, they'd be forming direct action groups, along the full spectrum of campaigning: all the way from awareness raising publicity campaigns, through picketing, blockades and sit-ins, up through Black Bloc type actions, right up to menacing intimidation campaigns and terrifying physical force operations. But I don't see any Butlerian Jihad getting started. Which tells me they don't actually believe this at all; they're just bigging up their own importance. 'Oh yes, our technology is so incredibly powerful, if it were done wrong then imagine what could happen! Keep the money coming to make sure it's done right instead! Then all that power can be ours instead!... I mean, uh, yours, Mr Investor sir.'

AI stock valuations don't make a bit of sense unless the technology turns out to be every bit as powerful as that. If they don't keep that thought alive, then the bubble bursts right now. That's what all this hot air is about, and that's why nobody really pulls a Miles Dyson at the AI research lab.

The interesting thing about the Terminator movies is that when AI researcher Miles Dyson became convinced that his work had a high probability of resulting in an artificial general intelligence attempting to replace humanity, he did not go and post a ten thousand word essay on LessWrong about how he had updated his timeline and p(doom) estimates and discussing the full Bayesian analysis of the situation. He went to the lab that very night with some heavily armed companions and he blew the place up.

I keep hearing that one AI researcher or another claims that they believe as Dyson came to believe. Until one of them takes similar action, I simply do not believe that they actually think their research carries such a risk.

You have access to the lab where the work is being done? You regularly meet in person with leading researchers and talents driving the project forward? You are an American and you have the Second Amendment? And the entire future light cone is at stake? Quintillions of hypothetical future lives riding on the outcome of this project here and now?

What's the most effective, altruistic thing you could do for them?

Yeah, exactly. I've never heard of anyone shooting up their AI lab. Which tells me they don't believe their AI is at all likely to wipe us all out.

... and then to secure Ukraine into the European economic and defence structure as firmly as possible. At this point they're far ahead of NATO on how to use drones and robots in war, and they're clearly getting very, very good at building them too. European military and aerospace people both will have a lot to gain from cooperation with Ukraine after the war is over.

As cadets were so memorably told a generation ago: remember always, your duty is clear - to build and maintain those robots!

Sounds good - DDG is pretty bad lately, it typically just returns a list of AI generated SEO slop pages, so I've been switching back to Google in a private tab for a lot of things. Might be worth trying this one now!

If they leave them there, the next administration might be able to switch them back on and start gathering woke climate science data again.