It matters quite a lot if someone sues you. Or if a propagandized public and their leaders get into a war over it.

The US Supreme Court, and supreme courts in general, are not "triggered." You appeal to them to hear your case. They may do so if they feel that your case is somehow relevant to wider issues.

The US Supreme Court might hear this case if they wanted to interpret the constitution in such a way as to set upper limits placed on fines, or wished to issue a judgement on the idea that fraud is still fraud if nobody gets hurt.

Yeah, that's a good example. Also "the actual thinking process as opposed to the correlational hallucinations that constitute all LLM 'thought'."

Two different sides of the same hubris.

A little tutorial if you know a little bit of Python:

https://numpy.org/numpy-tutori...

And a book with more detail if you're more into math:

https://www.deeplearningbook.o...

Everything else is pretty much scaling up and introducing some restrictions on the basic model.

You see it in the way you just used the word "you." Model "collapse" is a misnomer. It's drift, which you see in humans in literally everything they do, and you can demonstrate to yourself just by repeatedly generating random numbers and calculating the mean.

Bullshit. The fundamentals of modern AI models can be understood by anyone with a basic knowledge of algebra, which you should have picked up in junior high. They're piecewise linear approximations and use exactly the same equation as the linear regression you learned in high school or first year university. The more advanced stuff is hacky restrictions on that basic design to tone down the model's flexibility and make it easier to fit.

The reason it's hard to understand is because a) people who have no idea what they're talking about try and handwave their way through it; b) people explaining it want you to think it's really, really sophisticated and hard to understand, c) you're using "understand" in an unrealistic way or d) some combination of the above.

I kind of wonder if the best anti-distillation strategy is, if you detect suspicious traffic from someone (which happens a lot, they monitor for anything that looks like distillation), instead of blocking them, feed them say the output from Llama 3.1 8B or whatnot ;) Maybe finetune it a bit so it talks Claude-ish. But basically, subtly poison their dataset with hallucinations and crappy reasoning without it being immediately visibly obvious.

As for copyvio, sorry, this is something for the courts, and so far, the courts have not largely found against the trainers, and have instead found, by and large, that they're being compliant. The most notable setback against Anthropic for example was a finding that they couldn't just download books in training dataset off the internet, but that they could perfectly legally just buy surplus books for pennies on the dollar by the palletfull, scan them in, and train on that. That this is perfectly complaint with US copyright law.

I think a lot of you wish that copyright law was a lot more restrictive than it actually is. Which is a REALLY bizarre thing to see on Slashdot of all places, which back in the day was the beating heart of "Data Wants To Be Free!" philosophy.

To be clear, though... I would welcome a compromise modification to copyright law, which is, if you want to train on the public commons, you absolutely may, indeed, train on whatever you want, zero liability, but then you have to give back to the public commons. So maybe your top frontier models are closed, but you have to simultaneously release smaller distilled equivalent versions of it into the public domain (how to define "smaller distilled equivalent versions" is of course something that would require discussion), and release said frontier models to the public domain within e.g. 1 year or whatnot.

* They remain incentivized to keep pushing the frontier, since some people will always pay for the best
* They get permanently out of the worry of any copyvio liability (beyond basic requirements about not verbatim reproducing copyrighted materials in outputs)
* The public gets a constant stream of ever-better models, at no cost.

Sounds like a balance to me.

I think there is room to argue though, that distillation might violate copyright while training on large volumes of large material does not.

Nope. There is an argument about whether training a model on source material violates copyright or not. Current court decisions in the US say that the training does not, but you must have acquired the material legally in the first place.

Current court decisions in the US say that the raw output of a model is not copyrightable. Never mind that the people doing the distilling have paid for that output so even if it were copyrightable they'd own it.

The only thing happening is violation of terms of service which haven't been tested in court and would (hopefully) fail that test. Otherwise good luck with any software you use to produce anything, compilers included. And that's why these companies are directly lobbying the US government to do, uh, something, about it. They're trying to turn a bug in their business model into a geopolitical issue.

It doesnt think

Yeah, it does.

it doesnt rationalize

Yeah, it does.

These are not Markov chains. They're neural nets. They work via extremely complex chained fuzzy logic on superpositions of conceptual states.

And the less slop it has to deal with

This is literally a thread about distillation, aka, training on the outputs of other models. Synthetic data is the cornerstone of modern training. "Model collapse" is not something that actually happens in the real world, only in contrived settings, the model equivalent of if you could lock a person alone in a dark room with only their thoughts for ten thousand years.

What is "it" in this sentence?

Are you talking about the minuscule Google RAG model whose weights are probably measured in megabytes to lower-gigabytes and which you could run on a cell phone with capacity to spare, and comparing it to the multi-terabyte frontier models in question?

at least the person downloading such a component is responsible for "looking over it" before using it.

I'm sorry, but since when? You absolutely should not be tasking every single user of major projects with the responsibility of doing a code review on the whole thing before installing it. There needs to be a clear distinction between "fly-by-night thing that some rando uploaded" and "library that a million people depend on". In the traditional approach, the latter was something tat you installed with a package manner, while the former was something you went and fetch off e.g. Github or whatnot - and if you did the latter, then you were accepting that it was untrusted software. But now we have both (including the potential for "soundalikes") installed by the same means with no distinctions made by the install method. That is not good.

The funny thing about all this is that their only goal appears to have been to download publicly-available data off of UK government websites. I'm betting that they were (A) being tested on a knowledge-related task, and (B) not given direct access to the internet, but their tool capabilities involved access to the "gem" command (so that they could install Ruby packages for their work), and so decided to abuse (B) to cheat on (A).

Once again: training a model with the reward being "does it solve the task?" without looking at how it solves the task is very, very dangerous. This is basically the plot of Universal Paperclips.

I know a LOT of software that could use that as a pull request...

Of course the third bomb probably wouldn't have gotten used regardless of the order, as the next bomb (made from the infamous Demon Core) was scheduled to be ready to drop on the 17th, whereas Hirohito announced the surrender on the 15th (wasn't signed until 2 September, but it's hard to imagine that the US would have dropped the bomb after the announced surrender).

One of the amazing things is that the atomic bombings didn't shift the vote in the 3:3 deadlocked Japanese war council at all; the hawks (Anami, Umezu, Toyoda) remained focused on adaptation, not surrender. But it did make the doves (Suzuki, Togo, Yonai) more desperate, and Suzuki met with Hirohito. Hirohito had already been looking for an opportunity to surrender, having had plans to send Prince Konoe to Moscow to ask Stalin to function as a neutral broker to end the war (with secret instructions to accept "peace at any price"), but the trip was delayed by the Russian side, ostensibly by the Potsdam Conference (but in reality, to prepare for the invasion of Manchuria). When the terms of the Potsdam conference came out, Togo told Hirohito that they "were the most reasonable to be expected in the circumstances", to which Hirohito replied, "I agree. In principle they are acceptable." But the urgency to accept that in July wasn't present. Everything started collapsing in August, with the loss of their mainland and Pacific territories, the Soviet invasion, and the atomic bombings. What the bombings achieved was to help stir Hirohito to imminent action (which risked failing, as there was a coup attempt against him (the Kyujo Incident) to stop Japan from surrendering). The hawks saw unconditional surrender not as agreement to the loss of their sovereignty, but as agreement to enslavement.

But this is getting rather off topic. :) Main topic: if you think your AI is as much of a risk to the fate of humanity as the atomic bomb, which many of these people truly believe, then you can expect similar hesitancy as with the developers of the atomic bomb. They felt an urgent need - ending a brutal war - but at the same time had deep apprehension about what they were doing and how the fruits of their labour would be used.

And if you want a past example of scientists who feared their creation wanting to slow down, stop, or even undo it, BTW, let us consider the atomic bomb. Farrington Daniels took a poll of the Chicago nuclear scientists in 1945 asking what they wanted to happen with the atomic bomb. Remember that these were the people whose literal job was to make a weapon. Only 15% - 1 in ~7 - wanted the US to use it as it actually did, e.g. just start bombing cities just a couple days apart. Only a small majority wanted it even used in Japan, even against military targets without the Japanese first being invited to a demonstration in the US and being given ample opportunity to surrender. 13% - nearly as many as those who wanted the US to use it as it actually did - didn't want it used at all under any circumstance, in a war that had already caused a million American casualties.

The problem that they faced was that people like Groves** really wanted to bomb cities, and they, as the rank and file, didn't get a vote.

** - It seems dubious that Truman really understood what he was signing onto. While he never shirked from responsibility for authorizing it, the paper trail shows that he didn't seem to understand what he was approving at the time. In his diary before the attacks, he writes about the weapon, but about how he insisted it not be used against civilians, killing women and children, because he didn't want to sink to what he saw as the barbarism of the Japanese. In his first speech after the bombing, he refers to Hiroshima as "a military base", not a city, and says it was chosen to, insofar as possible, avoid the killing of civilians - but that if Japan did not surrender, then the US would bomb Japan's war industry, and then thousands of civilian lives would then be lost. Earlier drafts showed that it didn't contain the "insofar as possible" caveat. He genuinely didn't seem to realize what he was authorizing at the time. But Groves absolutely knew what he was doing, and deliberately selected sites for maximum civilian casualties. Truman seems to have come to understand what was going on by shortly after Nagasaki, where he gave the order to cease use of atomic weaponry, saying that the thought of wiping out another 100,000 people was too horrible and that he couldn't stomach the thought of "killing all those kids".