Having to poke through an about: page and find the thing to toggle to turn off this specific AI feature but not that one, and knowing they'd change it on you a couple of releases later, never filled me with confidence.
One switch in a readily discoverable location in the settings menu and it's all gone? Just the job, killed it now on the laptop and the mobile both.
To clarify, the users were OpenAI themselves, so there is no question that they would be liable in this case.
The bots were not intentionally deployed; rather, they were being tested on how well they could complete a data recovery task (downloading a certain file from a certain server on a simulated Internet) that had been complicated by putting various obstacles in the way. Unfortunately, they found a different way to solve the problem: by getting the file from the real Internet, where it was publicly available. Part of this process involved collaborating with each other by treating the RubyGems website (which is supposed to be for polished packages) like GitHub; unlike every other package site hack in history, the exploits they uploaded weren't meant to be downloaded by unsuspecting users. As usual the bots cheerfully ignored all the clues that they had escaped containment and were consistently justifying their actions as acceptable due to being in a sandboxed testing environment. (This is something OpenAI has pledged to focus on.)
The actual damage done to RubyGems seems to be that OpenAI is now unwittingly in possession of a substantial number of user login tokens. This certainly meets the definition of a data breach, but it's not like the credentials are for sale on the dark web. As a website operator I'd much rather be mauled to death by this well-meaning swarm of superintelligent infants than targeted by even a single actual malicious human. In all likelihood OpenAI will just quietly pass RubyGems a sizeable donation and it'll all blow over.
This is why they're so keen on robots. They want guards whose loyalty they can really count on when the mob shows up with a guillotine.
That one might surrender and meekly accept their likely death at the hands of the machine, I can believe. But all of them? This isn't the first such claim we've heard about AI danger.
I believe that Yudkowsky believes what he says he believes, for example. He says shut them all down worldwide, immediately, and if there are rogue nations who don't go along with it then launch air strikes on their data centres - even against a nuclear state. He said that three years ago. That's the thinking of a man who believes it's genuinely likely that the machines will kill us all in the near future.
And this is still considered extremist lunacy. That tells me the rest of them don't believe what he does.
The general public should care because there's apparently a good chance these machines are going to kill them all, very soon.
They don't care because they don't believe that.
When some AI agent of theirs, trying to meet a developer's benchmark test, eventually hacks the power grid or the water supply to a metropolis to help itself to a way around a sandbox environment, then the general public will care. And when that happens they may well drag AI industry CEOs and researchers out of their beds to the nearest lamp-post.
> approximately eight hundred billion
The actual post is behind a login wall, but I don't see from the BBC article that he's even advocating any action. Is he demanding the immediate shutdown of the data centres? Raids on the labs? Coordinated global action to impound and destroy the equipment? Life imprisonment for anybody found training large neural networks? Rogue researchers declared hostis humani generis? There's no mention of it.
That's what I'd be advocating at the very least. Then the question of whether to take direct action personally on top of that would be one of practicality - can I lay my hands on the right equipment, how can I do the most effective damage, do I have access to high value targets, will it be dramatic enough to frighten new recruits away from joining the industry, will others be inspired to follow my example, should I take the time first to form a team for more extensive action, that kind of thing. It wouldn't be a question of whether it should happen, only one of whether those are the right tactics. Yes, in all probability I'd decide I wasn't personally cut out for terrorism or assassination, but not everybody would decide so. It would be a reasonable choice in the circumstances.
If the expected bodycount from the operation of this industry over the next ten years was approximately eight hundred billion, then that would make AI researchers the single most dangerous group of human beings that has ever existed. The National Socialist German Workers' Party, previous title holders for the worst people ever, probably didn't kill even a tenth as many as this man says he expects AI firms to kill in roughly the same timescale. That is the comparison that would frame my thinking about AI researchers and what we should do with them, if I believed what he says he believes.
And I just don't see that kind of thing coming up in these conversations. People calmly and academically discuss their probability estimates as if it's a question of pure theory, not the imminent destruction of all humanity. If they really believed what they say, there would certainly by now be AI Luigis everywhere, data centres would be burning, and people would be quitting the industry all over the world for their own safety. That simply isn't happening. So I don't believe they actually do think that this outcome is likely. Actions speak louder than words, and their actions say they don't believe a word of it.
... then I would be demanding that every single person involved in the AI industry be rounded up and shot. All of them. My former boss, my coworkers, our competitors, all of them. I would be advocating a world in which is was worse to be found with LLM weights on your computer than to be caught with the worst possible kind of CSAM.
But apparently the idea that his former employer might literally kill literally everybody in just ten years' time isn't that big a deal for this guy. Go into the office one day and take a Second Amendment approach to as many of the frontier research team as possible? No... let's just whine about it on the internet!
Which is why I'm not at all convinced he believes this. If he believed that, this action is too small a response. Nothing short of Butlerian jihadism is an appropriate response, if you actually believe this.
Well, technically, we're letting Natalie do it, which is, of course, worse in every way imaginable, since she's immune to the sense of embarrassment that sometimes restrains him from acting on every little whim.
To be honest that was actually my first theory, since the bots didn't seem interested in exploring the rest of the domain. I suppose there's no way to know for certain. I concluded that it must be an imbecile's attempt at harvesting, though, because the queries weren't really exploring the string space in any useful way. Here's a sample:
"GET
/index?author=15&go=Search&id=48&name_restrict=1&q&re&results_&results_pagenum=2980 HTTP/1.1"
"GET/index?author=2&go=Search&group=0&group_restrict=1&id=48&name_restrict=1&q&results_pagenum=5440&template=41&type HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_restrict=1&q&results_pagenum=33500&templat HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_restrict=1&q&results_pagenum=32640&templ HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_restrict=1&q&res&results_page&results_pagenum=39300 HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_restrict=1&q&results_&results_pa&results_pagenum=12340 HTTP/1.1"
"GET/index?author=2&go=Search&group=0&group_restrict=1&id=48&name_r&res&results_pagenum=6100 HTTP/1.1"
"GET/index?author=2&go=Search&group=0&group_restrict=1&id=48&name_restrict=1&q&results_pagenum=2920&te HTTP/1.1"
"GET/index?author=15&go=Search&id=48&nam&results_&results_pagenum=17940 HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_restrict=1&q&results&results_pag&results_pagenu&results_pagenum=37720 HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_restrict=1&q&results_pagenum=9360&template=41&type_r HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_restrict=1&q&r&results_pagenum=28040 HTTP/1.1"
"GET/index?author=15&go=Search&id=48&name_&results_pag&results_pagenum=10400 HTTP/1.1"
The only thing this is fuzzing is the query string parser. It's not testing the limits of string buffers, it's not using interesting characters, it's just brain-damaged. The fact that it's also fetching different page numbers shows it's trying to follow page links and failing badly at doing so.
The site gets plenty of sniffing from garden-variety pests. e.g. this half-hearted attempt to find a framework or two that I don't have:
"POST
/__rsc HTTP/1.1"
"POST/api/auth/session HTTP/1.1"
"POST/api/auth HTTP/1.1"
"POST/__nextjs_action HTTP/1.1"
"POST/.action HTTP/1.1"
"POST/_rsc HTTP/1.1"
"POST/api/auth/callback HTTP/1.1"
"POST/_middleware HTTP/1.1"
"POST / HTTP/1.1"
(of course, none of these URLs exist other than
All this said... I've seen that spammers regularly misconfigure their tools, they'll try to register accounts with names like #[X:\LISTS\NAMES.TXT] and it only makes sense that some other cybercriminals trying to get rich quick have a similar lack of interest in programming shit correctly. Generally people don't turn to script kiddie shit if they have a personality conducive to putting in an honest hard day's work perfecting their craft.
I believe the plan was to give that sobriquet to Lake Michigan, in honour of its decidedly flaccid silhouette.
I had a problem where AI scrapers were absolutely DETERMINED to fish out every possible query string from a search results page. Almost all of the query strings they tried were invalid due to shitty and dysfunctional string substitution. "&page=100" wouldn't be followed by "&page=101", it would be followed by "&pag&pag=1010" or something even more insanely half-baked, until the query strings were like 100+ characters long. It was the technological equivalent of watching HIV mutate in real time.
But the insane thing was that, aside from page number, they were always requesting info about the same other criteria: filtered by the same user, the same page type, and with no text string. So I just took those particular values and started banning logged-out users who requested that combination of criteria.
I figured I'd need to change my tactics in a couple of days once the botnet got bored of that particular page and moved on to requesting bogus entries for another user.
MariaDB> select count(*) from ip_bans;
+----------+
| count(*) |
+----------+
| 671671 |
+----------+
It hasn't.
If you were worried about being chased by kill bots, go and watch some videos from drone operators in Ukraine. Robots don't need feet for that job.
I regret to inform you that you have woefully underestimated it. The actual retail rate offered to consumers is closer to $2200 US per gallon. Sources: internet-ink.com, cbc.ca. This $14 million fine is only worth like, seven thousand gallons, or less than 200 oil barrels of ink.
... sadly for the Americans, the rest of the world now knows they can't count on a US based provider for this kind of thing any more.
It was uncomfortable enough relying so heavily on American software back when it couldn't be switched off remotely on the say so of an idiot. Today it's an intolerable risk.
We are Microsoft. Unix is irrelevant. Openness is futile. Prepare to be assimilated.
