Learn to read dick

A reporter following a person working for a surveillance company is not in fact unusual. You lie every time you speak.

Lol reliability. Both furnaces in this house needed not just new generators but also gas valves. One of them had been replaced before.

Randomness is the problem in LLMs that leads to inconsistency of hallucination. It's being substituted for the part of the processing we do that software can't do yet. Hardware can't do it either, only wetware. The part that (inductively? Instinctively?) knows whether the output makes sense just plain doesn't exist in LLMs.

I kind of wonder if the best anti-distillation strategy is, if you detect suspicious traffic from someone (which happens a lot, they monitor for anything that looks like distillation), instead of blocking them, feed them say the output from Llama 3.1 8B or whatnot ;) Maybe finetune it a bit so it talks Claude-ish. But basically, subtly poison their dataset with hallucinations and crappy reasoning without it being immediately visibly obvious.

As for copyvio, sorry, this is something for the courts, and so far, the courts have not largely found against the trainers, and have instead found, by and large, that they're being compliant. The most notable setback against Anthropic for example was a finding that they couldn't just download books in training dataset off the internet, but that they could perfectly legally just buy surplus books for pennies on the dollar by the palletfull, scan them in, and train on that. That this is perfectly complaint with US copyright law.

I think a lot of you wish that copyright law was a lot more restrictive than it actually is. Which is a REALLY bizarre thing to see on Slashdot of all places, which back in the day was the beating heart of "Data Wants To Be Free!" philosophy.

To be clear, though... I would welcome a compromise modification to copyright law, which is, if you want to train on the public commons, you absolutely may, indeed, train on whatever you want, zero liability, but then you have to give back to the public commons. So maybe your top frontier models are closed, but you have to simultaneously release smaller distilled equivalent versions of it into the public domain (how to define "smaller distilled equivalent versions" is of course something that would require discussion), and release said frontier models to the public domain within e.g. 1 year or whatnot.

* They remain incentivized to keep pushing the frontier, since some people will always pay for the best
* They get permanently out of the worry of any copyvio liability (beyond basic requirements about not verbatim reproducing copyrighted materials in outputs)
* The public gets a constant stream of ever-better models, at no cost.

Sounds like a balance to me.

It doesnt think

Yeah, it does.

it doesnt rationalize

Yeah, it does.

These are not Markov chains. They're neural nets. They work via extremely complex chained fuzzy logic on superpositions of conceptual states.

And the less slop it has to deal with

This is literally a thread about distillation, aka, training on the outputs of other models. Synthetic data is the cornerstone of modern training. "Model collapse" is not something that actually happens in the real world, only in contrived settings, the model equivalent of if you could lock a person alone in a dark room with only their thoughts for ten thousand years.

I thought "Mommy Has Four Graphics Cards" was pretty good, but "Not Everyone Computes" did a great job of explaining why the industry is largely about remote services rather than running it locally.

Cowards who only have whataboutism are my primary detractors, which is how I know I'm right. Thanks, coward.

The question at hand is whether the terms of service should even be allowed to contain those clauses. If you're paying for access to the model, it should be your business what you use the output for. If they don't want you using the data you got out of it, they can not sell you access to it in the first place.

Anyone mad about this should note that $50,000 per year is what In-n-Out Burger pays a FT employee in California, and then they should fuck off to the farthest possible point to fuck off to, and then fuck off some more.

How does one become a high-end skilled worker? Clown.

You don't know shit about logic, son, you only know emphasis.

There are no innocent workers installing surveillance equipment. Zero.

Private prisons are the worst but they aren't the only problem prisons. A government employee's "power" is based on the headcount of their reports. More problems, more headcount, more power. More opportunities for kickbacks and to pocket funds.

Is it incidental, or is it the system protecting itself? Recidivism leads to profit.

Since the license disclaimed responsibility.

And yet that is the current, prevailing situation. There is no warranty, so you are responsible, full stop. But this is true for closed commercial software as well. If they are exploited and consequently send you malware, it's your problem.

There hasn't been since we moved away from the cathedral model, but you couldn't trust the cathedrals either.

The serious things get integrated into distributions and you can get them that way. If you choose to get them another way in order to get a newer version or a fork, you are taking an additional risk. Again, this is how the bazaar model has always worked. This is not new.