I always figured that people who genuinely believed their employers were likely to kill literally everybody in the near future might burn down a bit more than that, but so far it's mostly been tweets

Having to poke through an about: page and find the thing to toggle to turn off this specific AI feature but not that one, and knowing they'd change it on you a couple of releases later, never filled me with confidence.

One switch in a readily discoverable location in the settings menu and it's all gone? Just the job, killed it now on the laptop and the mobile both.

This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.

That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.

I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.

And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.

The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.

They don't care because they don't believe that.

When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.

> approximately eight hundred billion ... oops, that's eight hundred million, of course. Ten percent probability, times the current population of the Earth. The expected consequences of allowing the AI industry to continue their work, according to people who are oddly calm about the prospect.

The actual post is behind a login wall, but I don't see from the BBC article that he's even advocating any action. Is he demanding the immediate shutdown of the data centres? Raids on the labs? Coordinated global action to impound and destroy the equipment? Life imprisonment for anybody found training large neural networks? Rogue researchers declared hostis humani generis? There's no mention of it.

That's what I'd be advocating at the very least. Then the question of whether to take direct action personally on top of that would be one of practicality - can I lay my hands on the right equipment, how can I do the most effective damage, do I have access to high value targets, will it be dramatic enough to frighten new recruits away from joining the industry, will others be inspired to follow my example, should I take the time first to form a team for more extensive action, that kind of thing. It wouldn't be a question of whether it should happen, only one of whether those are the right tactics. Yes, in all probability I'd decide I wasn't personally cut out for terrorism or assassination, but not everybody would decide so. It would be a reasonable choice in the circumstances.

If the expected bodycount from the operation of this industry over the next ten years was approximately eight hundred billion, then that would make AI researchers the single most dangerous group of human beings that has ever existed. The National Socialist German Workers' Party, previous title holders for the worst people ever, probably didn't kill even a tenth as many as this man says he expects AI firms to kill in roughly the same timescale. That is the comparison that would frame my thinking about AI researchers and what we should do with them, if I believed what he says he believes.

And I just don't see that kind of thing coming up in these conversations. People calmly and academically discuss their probability estimates as if it's a question of pure theory, not the imminent destruction of all humanity. If they really believed what they say, there would certainly by now be AI Luigis everywhere, data centres would be burning, and people would be quitting the industry all over the world for their own safety. That simply isn't happening. So I don't believe they actually do think that this outcome is likely. Actions speak louder than words, and their actions say they don't believe a word of it.

... then I would be demanding that every single person involved in the AI industry be rounded up and shot. All of them. My former boss, my coworkers, our competitors, all of them. I would be advocating a world in which is was worse to be found with LLM weights on your computer than to be caught with the worst possible kind of CSAM.

But apparently the idea that his former employer might literally kill literally everybody in just ten years' time isn't that big a deal for this guy. Go into the office one day and take a Second Amendment approach to as many of the frontier research team as possible? No... let's just whine about it on the internet!

Which is why I'm not at all convinced he believes this. If he believed that, this action is too small a response. Nothing short of Butlerian jihadism is an appropriate response, if you actually believe this.

If you were worried about being chased by kill bots, go and watch some videos from drone operators in Ukraine. Robots don't need feet for that job.

Whether we like it or not, whether we use Windows or not, Microsoft is going to insist on adding the tax whether the PC comes shipped with Windows or not. This is illegal bundling, Microsoft has essentially ignored prior rulings, and it's pretty much a certainty the courts will be fine with this.

"How did you get hired?"
"I was top at Fortnite. You?"
"Combat flight sims."

"Damn, two Boeings just crashed over an oil refinery on city limits and took out half the city."
"Did you remember to save your position beforehand?"
"Yeah."
"The reload and continue from there. No-one will notice."

I love computer games. I have XPlane 12 and many scenery packs. I rank well on Elite:Dangerous. From the sounds of it, the FAA would see me as over-qualified. In reality? There's no way in hell I'd be taking those kinds of risks with real lives. There's a huge difference between having good reflexes and a good eye, versus having the complex 4D spacetime relationship mental models needed for robust air traffic control.

You see, Linus has embraced transhumanism and is testing out the new kernel as a supplementary brain function, using MOSIX to offload all of the irritable comment generation at yet more nonsense on the mailing list to an Elizabot that he has written specifically to do this. This saves his actual brain for real work.

AI is not currently capable of performing any meaningful conceptual abstraction, it is only capable of very basic mechanical abstraction. Abstraction is itself multi-dimensional. Nor is there any indication that AI could ever perform multi-dimensional abstraction or multi-dimensional decomposition. Precious few humans are capable of it either, but some can.

No, AI would need humans because the best system is not a pure system but a hybrid system, and that means transhumanism with AI operating as an additional brain function rather than as a replacement for a brain.

Tool calling is, yes, but the current approach to it is (a) overt not transparent to the user, (b) not designed for the purpose I've outlined, and (c) not remotely good enough for the AI to be able to manage data through such tools.

Yes, you can connect an AI to a PostgreSQL database. Whoopee. Not even close to an AI detecting via classifiers that some of the data is relational in nature, transparently setting up its own PostgreSQL database in response, and using that proactively as an additional way to examine the data. You would need to explicitly set up the constructs, explicitly set up the databases, explicitly tell it when, where, and how to use the database, and if that particular usage conflicted with a way the AI actually worked, the AI would not be able to use it effectively. That's not remotely close to what I'm talking about.

What I'm talking about is much, much deeper than that. AIs lose focus when there is too much information currently in the system. Absolutely nothing stops an AI from using a document database as a virtual memory in which it can page in and out conceptual spaces so that the focus on any given step of a problem concerns just the problem. Other than such a concept not existing yet, which is kind of a limiting factor. But you need to know what connects to what. The AI could use an ontology reasoner for that, or a relational database, or an external graph. But you cannot provide those tools and you cannot configure them, for the simple reason that the AI is the only entity with any detailed map of how the underlying neural net is connecting those ideas up.

The AI itself has to select the tools, has to configure the tools, has to do all the work. Which means YOU cannot use any sort of API. YOU should have no involvement in the underlying mechanisms needed for the internal NN housekeeping.

And that's your other error. You're assuming this is about user data. No. It's about housekeeping operations within the NN itself to maximise functionality, it has nothing to do with the user side of things at all. If the user was capable of setting up dynamically structured databases that mutated with each step of a decomposed analysis, you'd have done everything the AI would be doing and you wouldn't need the AI to begin with.

You cannot do this work, you cannot even assist in this work, it has to be dynamic and it has to be utterly invisible to the operator.

We don't need bigger models, at this point. What we need is multi-dimensional decomposition, problem space transforms, and the ability for AI to use external tools (such as SQLite, memcached, etc) so that it can externalise static data that it needs to not corrupt accidentally but still keep in easy access.

If we had that, most of the things "bigger models" will do will actually end up being done better, quicker, with fewer compute resources.