Having to poke through an about: page and find the thing to toggle to turn off this specific AI feature but not that one, and knowing they'd change it on you a couple of releases later, never filled me with confidence.
One switch in a readily discoverable location in the settings menu and it's all gone? Just the job, killed it now on the laptop and the mobile both.
This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.
That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.
I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.
And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.
The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.
They don't care because they don't believe that.
When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.
> approximately eight hundred billion
The actual post is behind a login wall, but I don't see from the BBC article that he's even advocating any action. Is he demanding the immediate shutdown of the data centres? Raids on the labs? Coordinated global action to impound and destroy the equipment? Life imprisonment for anybody found training large neural networks? Rogue researchers declared hostis humani generis? There's no mention of it.
That's what I'd be advocating at the very least. Then the question of whether to take direct action personally on top of that would be one of practicality - can I lay my hands on the right equipment, how can I do the most effective damage, do I have access to high value targets, will it be dramatic enough to frighten new recruits away from joining the industry, will others be inspired to follow my example, should I take the time first to form a team for more extensive action, that kind of thing. It wouldn't be a question of whether it should happen, only one of whether those are the right tactics. Yes, in all probability I'd decide I wasn't personally cut out for terrorism or assassination, but not everybody would decide so. It would be a reasonable choice in the circumstances.
If the expected bodycount from the operation of this industry over the next ten years was approximately eight hundred billion, then that would make AI researchers the single most dangerous group of human beings that has ever existed. The National Socialist German Workers' Party, previous title holders for the worst people ever, probably didn't kill even a tenth as many as this man says he expects AI firms to kill in roughly the same timescale. That is the comparison that would frame my thinking about AI researchers and what we should do with them, if I believed what he says he believes.
And I just don't see that kind of thing coming up in these conversations. People calmly and academically discuss their probability estimates as if it's a question of pure theory, not the imminent destruction of all humanity. If they really believed what they say, there would certainly by now be AI Luigis everywhere, data centres would be burning, and people would be quitting the industry all over the world for their own safety. That simply isn't happening. So I don't believe they actually do think that this outcome is likely. Actions speak louder than words, and their actions say they don't believe a word of it.
... then I would be demanding that every single person involved in the AI industry be rounded up and shot. All of them. My former boss, my coworkers, our competitors, all of them. I would be advocating a world in which is was worse to be found with LLM weights on your computer than to be caught with the worst possible kind of CSAM.
But apparently the idea that his former employer might literally kill literally everybody in just ten years' time isn't that big a deal for this guy. Go into the office one day and take a Second Amendment approach to as many of the frontier research team as possible? No... let's just whine about it on the internet!
Which is why I'm not at all convinced he believes this. If he believed that, this action is too small a response. Nothing short of Butlerian jihadism is an appropriate response, if you actually believe this.
It's a way of reframing sloppy practices as a signal to your investors that your software has become powerful. Most of the horror stories about AI showing signs of becoming hyperintelligent are coming from the companies themselves, which should tell us something.
While the energy source of wind and solar are free, building and maintaining the plants is not free.
If all they were building were a single 100MW data center, sure: renewables would be cheaper to build. But they evidently are planning a major campus that will consume most of the plant's 615 MW output over the next 25 years.
A solar installation, in a favorable location, capable of supplying 600MW around the clock (with battery backup) would have to be roughly 1800MW in capacity. It would be among the largest inthe world, on the order of ten thousand acres in size at a build cost of maybe 2 billion. The battery backup system to ensure 99.9% uptime would be 9x the size of the largest li-ion grid storage system ever built, and set you back on the order of 4-5 billion dollars.
While it's probably cheaper to go renewable than build a *new* nuclear plant, if you can reactivate and one for just two billion that looks like a bargain, if you have a use for all that power. Inability to save money by load following is the financial Achilles' heel of this generation of reactors, but if you have a guaranteed customer for most of your output, years in advance, that's as close to an ideal economic case for them as anything could be.
ToS is the strongest argument, but the distillers are not parties to the ToS. They get their data from data brokers. It is possible that data brokers are violating the ToS, but it would be hard to write ToS that precluded running queries for third parties without creating problems for consultancies and other businesses. Even presuming the ToS could be written to preclude the data brokers doing that, it doesn't affect the resulting model.
But the general shape of the argument brings us right back to unclean hands: we worked hard on this model and it's not fair for you to profit off our work in a way that doesn't have our permission.
I think the fair-minded position is to see what people think when the documentary comes out. You're essentially arguing a negative here -- that there *can't* be anything of value that hasn't been said yet.
Also, I don't think saying something *new* is necessarily critical. Sometimes saying something obvious but in an interesting way is worthwhile.
In this case, Holmes gave access to the filmmaker. Obviously she has an agenda. If the filmmaker is smart, he has an agenda that's different from hers. If he knows his business he'll find something interesting to show you from that conflict.
Now looking at the trailer -- holy shit that woman gives off batshit crazy vibes. That's new to me. I expected her to be slick, persuasive, like Saruman in Lord of the Rings: someone. you'd need real strength of mind to resist being persuaded by. If I'd just handed someone like that my business card, I'd get it back on some pretext then head as fast as I could for the door.
It doesn't make it *right*, but it does make claiming it is *wrong* inconsistent with their own behavior. This could prevent the US companies from suing the Chinese companies seeking an injunction (due to the "unclean hands" doctrine), and probably blocks them from seeking monetary damages in most US jurisdictions.
And suing may undermine the US companies own intellectual property claims by exposing their shaky foundations. An AI model isn't *expression*, so it can't be copyrighted. Insofar as the service allows the underlying model to be deduced through regular usage, trade secret protections don't apply because that's *reverse engineering*, which trade secrets don't prevent.
This leaves violations of terms of service. But setting aside the dubious enforceability of anti-reverse engineering provisions, the Chinese companies may not even be parties to the ToS agreement if they are obtaining the model data through a network of contractors and shell companies.With only moderate paranoia, they can effectively shield themselves from some kind of US tort claim.
Which leaves them with the model they've created from the US company's model, but which the US company has no IP claim upon. If I download a DeepSeek model that's been (hypothetically speaking) trained on Claude, Anthropic might not like that, but what can they do about it?
It's a strange situation. US investors are spending cumulatively on over half a trillion dollars a year in the hope of owning a breakthrough frontier model that is the end of the economic world as we know it. But the essence of the model might not be intellectual property at all.
Think of the categories as fixed, and the people moving *through* them. Over 65s are exiting the category at a lower rate.
It's more a case of the fox being in charge of the chicken coop; or bank robbers gaining control of the bank.
The job of the press in a democracy is to hold officials up to scrutiny.
"If it's not loud, it doesn't work!" -- Blank Reg, from "Max Headroom"
