This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.

That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.

I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.

And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.

The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.

They don't care because they don't believe that.

When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.

> approximately eight hundred billion ... oops, that's eight hundred million, of course. Ten percent probability, times the current population of the Earth. The expected consequences of allowing the AI industry to continue their work, according to people who are oddly calm about the prospect.

The actual post is behind a login wall, but I don't see from the BBC article that he's even advocating any action. Is he demanding the immediate shutdown of the data centres? Raids on the labs? Coordinated global action to impound and destroy the equipment? Life imprisonment for anybody found training large neural networks? Rogue researchers declared hostis humani generis? There's no mention of it.

That's what I'd be advocating at the very least. Then the question of whether to take direct action personally on top of that would be one of practicality - can I lay my hands on the right equipment, how can I do the most effective damage, do I have access to high value targets, will it be dramatic enough to frighten new recruits away from joining the industry, will others be inspired to follow my example, should I take the time first to form a team for more extensive action, that kind of thing. It wouldn't be a question of whether it should happen, only one of whether those are the right tactics. Yes, in all probability I'd decide I wasn't personally cut out for terrorism or assassination, but not everybody would decide so. It would be a reasonable choice in the circumstances.

If the expected bodycount from the operation of this industry over the next ten years was approximately eight hundred billion, then that would make AI researchers the single most dangerous group of human beings that has ever existed. The National Socialist German Workers' Party, previous title holders for the worst people ever, probably didn't kill even a tenth as many as this man says he expects AI firms to kill in roughly the same timescale. That is the comparison that would frame my thinking about AI researchers and what we should do with them, if I believed what he says he believes.

And I just don't see that kind of thing coming up in these conversations. People calmly and academically discuss their probability estimates as if it's a question of pure theory, not the imminent destruction of all humanity. If they really believed what they say, there would certainly by now be AI Luigis everywhere, data centres would be burning, and people would be quitting the industry all over the world for their own safety. That simply isn't happening. So I don't believe they actually do think that this outcome is likely. Actions speak louder than words, and their actions say they don't believe a word of it.

... then I would be demanding that every single person involved in the AI industry be rounded up and shot. All of them. My former boss, my coworkers, our competitors, all of them. I would be advocating a world in which is was worse to be found with LLM weights on your computer than to be caught with the worst possible kind of CSAM.

But apparently the idea that his former employer might literally kill literally everybody in just ten years' time isn't that big a deal for this guy. Go into the office one day and take a Second Amendment approach to as many of the frontier research team as possible? No... let's just whine about it on the internet!

Which is why I'm not at all convinced he believes this. If he believed that, this action is too small a response. Nothing short of Butlerian jihadism is an appropriate response, if you actually believe this.

Every time people hear what it actually involves is outrage and opposition, at least all the way back to when Intel first announced that they wanted to hardcode identity numbers inside CPUs in 1999. But the corporations involved have been relentless, and have sunk countless billions of dollars steadily forcing it forwards, and building front organizations to obfuscate and whitewash it.

With Windows 11, Microsoft was finally able to FORCE Trusted computing hardware into every new computer. Not just every new Windows computer, but every computer.

Every new Intel PC CPU has built in Trusted Computing enforcement hardware. Every new AMD PC CPU has built in Trusted Computing enforcement hardware. Every new ARM PC CPU has built in Trusted Computing enforcement hardware. The only processor lines that DON'T have it are the microcontrollers.

There are some unlocked smartphones available, but as far as I can determine it's literally impossible to buy a smartphone that doesn't have hardware trusted computing built in.

I specified that enforcing Trusted Computing at the internet access level is still a long term goal, they couldn't get away with it today. However they are well on the way to success. Virtually all new hardware supporting trusted computing, all the front groups rolling out standards and systems, and as it gets incorporated into things everyone is going to increasingly running into situations where they get locked out of stuff if they're not Trusted Computing compliant. Streaming already restricts you to the worst quality if you're not compliant, and it's only going to get worse as pre-Win11 computers fade out and as more things require it.

You are entirely correct. I took another look at what the law actually required and I am definitely crying wolf (at this point). Thanks for taking the time to point this out to me.

There is definitely a slippery slope there, but I agree with you, California is far enough away from the event horizon of that slope that my histrionics are not warranted. Not only that, but the carve out for Linux actually is helpful. Thanks once again.

I'd hardly call exact date of birth "low information content".

And yes this does reveal exact date of birth, regardless of the bullshit obfuscation that it supposedly only reports age range. The server simply tracks the reported result every time the user connects, and on some specific day the result CHANGES to announce their date of birth.

-

This is based on SLSA (Supply-chain Levels for Software Artifacts), brought to you by the same fuckers making Trusted Computing and the TMP (Trusted Platform Module). It's part of the same shitstack to prohibit you from altering your software and to lock you out of your own files, and to send spy reports out over the internet so you can be cut off if you "fail" the Trusted Computing check.

The software can use a TPM's (Trusted Platform Module) Sealing function to encrypt your data such that it's impossible to access your own data if the software is modified. It can then pass control over that data only to software updates that carry a signed security certificate from Broadcom (or any other company using this system).

It is no longer open source, your system no longer works as you can no longer access your Sealed data if you change so much as a single letter of the code. Even recompile unaltered code won't work, unless you magically manage to get your build environment absolutely identical to the company's build environment and get byte-for-byte output. Even that may be impossible with the newer levels of non-deterministic compiler optimizations.

Also, with TMP's Remote Attestation feature can be used to transmit your machine's software configuration over the internet, so that you can be cut off if your system doesn't match Broadcom (or other company's) cryptographically signed certificate.

And then of course there's Network Access Control (NAC) / Trusted Network Connect (TNC). In the long term, the goal is for ISPs to use NAC/TNC to interrogate your computer for Trusted Computing compliance, and deny you any internet access whatsoever if your machine isn't compliant. Software with this sort of "security" certificate would pass inspection, while any attempt to alter the code would be detected as "tampering". You then get "quarantined". What "quarantine" means is that you are denied internet access - with the exception that you do get very restricted access which can only be used to download the approved software to "fix" your computer into Trusted Computing compliance.

The companies in question are going to get in serious legal trouble if they fail to comply in verifying the ages of their users. Microsoft, Google, and Apple are all going to give these companies an API that will move that burden from themselves to the makers of the operating system. Discord, to use an example, will be able to say that they asked Microsoft, Google, and Apple to verify ages, and then that they relied on the information that they were given. What's more these companies actually want to know the age of their users. This information is valuable, and they will be able to use it to target their customers with advertising.

What's more, every bad actor is going to try and target this exception. Linux traffic will end up in the same bucket as bots, predators, and any other malicious traffic. There is nothing in the current exception that requires companies to work with Free Software. My guess is that if your operating system can't do age verification that is blessed by Microsoft, Google, or Apple that most sites will simply block your traffic.

This is going to end up being a huge step backwards for Free Software clients.

Social media sites are going to get in trouble if they can't verify ages, and they want the age information so that they can sell it to advertisers. Microsoft, Apple, and Google are going to create APIs that the various services are going to trust. If your alternative implementation of the API response isn't signed by a key controlled by Microsoft, Apple, or Google it will be blocked outright. This loophole is not a victory, it is a huge defeat.

These operations can get in actual legal trouble if they don't gather this information. Plus, it is in their best interests financially to gather this information. Verified age information is ridiculously valuable to them. They want to be able to get your verified age, and now they have a legal reason to require it. Plus, every bot, actual predator, and other bad actor is going to try and climb through this Free Software exemption. The overwhelming majority of traffic that can't provide a response signed with a key by Microsoft, Apple, or Google is going to be crap, pure and simple. This law doesn't say anything about requiring that the service work with Free Software. Instead it puts Free Software in its own little ghetto. One thing is certain about ghettos, they don't put you in a ghetto for any reason but to be able to discriminate against you.

It is far more likely that a year from now Linux will be unusable for sites that require age verification, than it is that Linux will be the loophole that websites allow ti work despite not having "certified" age verification. Microsoft, Apple, and Google didn't fight this because they know that this actually strengthens their grip on operating systems. From now on, any device created by people that want to potentially do business in California (ie, everyone), is going to have to find a way to get a "blessed" age verification API by one of the big three. If they can't get that blessing then their fancy new device is unlikely to work on any service that has personalized accounts.

The exception only gives websites and other software a way to break if you insist on using Free Software. Every malicious bot, every child predator, every last bad actor is going to pretend to be Free Software, and people needing to comply with the law are simply going to block everything that isn't made by Google, Microsoft, or Apple. The reason that this got through the legislature is that it is the opposite of a win for Free Software, it is a devastating loss.

The commercial software vendors are going to create a framework that is going to insist on software signed by one of them to be "legitimate," and everyone that needs this verification to meet the standards of the law is going to simply block traffic that doesn't provide it. Free Software will still technically be legal, but it won't interoperate, and so it won't be usable.

You can see a bit how this works with Google Chrome on Linux. It mostly works great, until you need to watch something that requires Widevine, and then it either doesn't work at all, or it works with artificially reduced functionality. This is not because Chrome on Linux is less secure, or less capable, but rather because it is different. This isn't stopping piracy, the streams still escape. In fact, there are streaming services that I don't pay for, despite wanting to watch their content, simply because I would have to boot into something besides Linux to use their service, and the extra work isn't worth the hassle.

The difference is that this law is going to cover essentially every service that we want to use on the Internet. Every website, every app, every service with accounts that wants to be able to do business in California (and that's basically everyone), is going to require this age verification. Not only are they going to get in trouble if they don't get this information, but they actively want the information so that they can use the information to sell to advertisers. These operations aren't going to work on a non-validated code path. They are simply going to require commercial software verification and block everything else. It is not only the path of least resistance, it is the path of greatest profit.

No where does the law require that services work equally well without age verification. No where does the law say that the standard has to be open and implementable by Free Software. The reason that this exception stayed in the final bill is that the commercial software vendors saw this and realized that if they were going to be forced to build this API they could at least use it as an opportunity to shoulder out their competition for the long term.

Heck, this doesn't just work against Free Software. It is going to work against any newcomer to the party. Anyone creating a new device is going to have to use negotiate to use software blessed by Microsoft, Apple, or Google, because website and application operators are going to be skeptical of anything else. Anything else is going to look like a bot, or an attack, or some other bad actor.

The new system is simply another set of inputs. You will still get search results even if you don't click on anything. In fact, I would bet that actual user interaction with this new system is going to be very small. Most of the input that Google will get will be people trying to game the system.

I didn't even know about Kagi. In fact, it has been a long time since I have even considered using someone besides Google. I just resigned myself to the idea that things were forever broken. Thanks for the insight.