Forgot your password?
typodupeerror

Comment Re: all written from scratch? (Score 1) 72

You're an idiot.

If you anthropomorphize the term "learn", then you're correct. They learn nothing.
But then again, at that point, a heuristic spam filter learns nothing either, and yet we have called that learning for 30 years.
So, accepting that this level of flexibility is allowed for the word "learn", we can accept that the backpropagation is, in fact, learning.

Further, "learning" is the exact fucking opposite of novelty. It pains me that people as stupid as you have the boldness to share your half-baked opinions.

Comment Re: all written from scratch? (Score 1) 72

Absolutely incorrect.
It is a statutory exception to copyright.
Where my fair use rights intersect with your copyrights, the rights granted by your copyright are null and void- and that's clearly written in the statute.

That part that is not clear, of course, is whether or not my opinion about whether something is fair use or not, is actually fair use- and from that perspective, you can say it's a defense, since I'll have to make that argument into a defense if a prosecutor doesn't buy it.

Comment Re: all written from scratch? (Score 1) 72

Agreed absolutely. However, as mentioned by me, if as described by the GP was their incorrect reduction of what an LLM does, then that question is absolutely still in flight, because Judges aren't morons who can be talked into seeing waterboarding as "essentially washing your face".

Comment Re: all written from scratch? (Score 1) 72

but your fair use doesn't include publishing it.

Again, not necessarily.
Every country I'm aware of contains fair use provisions for transformative use. It would be strange if they didn't.

I answered with "not necessarily" because it really does have a lot more to do with the physical act of "taking your image and editing it with photoshop", not that that accurately describes how an LLM works at all.

This isn't to defend the pro-LLM argument by any means, but it is the argument, and it has succeeded in some courts, and failed in others.

Comment Re: all written from scratch? (Score 1, Flamebait) 72

Cool, and my grandma's goal could have very well been to win the Tour De France, yet it still didn't make her any more bicycle-like.

Your analogy is that your grandma wanting to achieve something unachievable for her is comparable to LLM pretraining specifically trying *not* to memorize token sequences?
Not very intelligent, are you?

Do you have any proof of that?

Do I have proof of that?
It's self-fucking-evident. Do you think a calculator decodes an unwrapped ALU output to answer your inputs?
But guess what- it could.

Because vast majority of people definitely see various forms of "1 + 1 = 2" repeat many times before they get to the level where they can derive that formula themselves, and I'd argue it's fairly easy to prove that vast majority of people actually remember that "1 + 1 = 2" and don't derive it in practice.

Strawman. Probably unintentional on your part given your demonstrated inability to hold a train of thought, but:
I asserted that I did not need to, you answered with "well I bet most people do."
If the simplified example was confusing, replace 1 + 1 = 2 with 44321 * 454531 = 20145268451. Yes, an LLM can solve that, and no, it has never seen it in its training data.

As far as I know, we have seen LLMs generally tending towards being more reliable at repeating things that are already in the data than they are at inventing novel things

LLMs are terrible repeaters, except for very contrived situations.
The fact that their knowledge base is centered around the exact data they have ingested is not a mystery. That being said, models generalize- and that is the goal of pretraining- for them to generalize. Repating is an error that you want to avoid, because a model that repeats is a dumb model.
We don't want the model to know that Paris comes after "What is the capital of France?". We want it to have encoded Paris as a thing called a capital of a place that is called france that is a country in europe. And they do.

and as far as I know, there's been zero LLMs that have written operating systems "from scratch" that haven't had multiple complete operating systems in their training data.

It would indeed be very unlikely that an LLM that had not seen examples of an operating system could ever generate one. That doesn't mean it reproduced what it learned. It means it learned the shape of an operating system from its data.
Again, I do not need to encode 44321 * 454531 = 20145268451 to solve for 44321 * 454531.

I've been training neural nets since well before the initial Transformers paper.

Perhaps you have, but in the same way an LLM would- you had no fucking idea what you were doing ;)

You vomiting a couple [overfitting mitigation methods] isn't impressive to me.

I suspect nothing impresses a bullshit artist such as yourself- it's part of your persona to hide that.

We've known for a long time that some information is more useful when blurred, and often easier to memorize when less specific. Yes, of fucking course, we're aiming for generalization in AI models.

And your regurgitation of things you don't understand isn't impressive to me.

But I you haven't shown anything to prove that this particular LLM isn't just sticking together pieces of kernels it has seen. Nobody calls a collage "from scratch".

It's quite easy to prove that this doesn't contain a single line of linux within it. Nobody said it doesn't contain the semantic concepts that it learned from linux- that was, in fact, the entire point you dim-witted poser.
The models knows what goes into a scheduler. It can reproduce a scheduler with any kind of variation you need within the shapes of other concepts it has learned. It does not need the code it took to train its weights to understand that shape.

Nobody calls a collage "from scratch".

And nobody calls someone else's code a collage, though by your definition, it most certainly is.

Comment Re:It's an interesting metric (Score 1) 39

I think that is ultimately what the industry is trying to figure out right now.
It's certainly cheaper to use tokens in a subsidized model, but the subsidized model is bullshit- so there's that. They can't keep selling the tokens at a loss forever.
At raw API prices, a human is flatly cheaper for this kind of work, and likely more reliable.
Of course the flip side of that, is that the inference itself isn't what's expensive, it's the model arms race, which likely isn't sustainable in any scenario.

Comment Re: all written from scratch? (Score 0) 72

No.

You are of course correct that you could train a model to try to reproduce outputs exactly, but that is not how it is done, because that will produce an absolutely terrible LLM.
The goal is to capture the semantic meaning, not the sequence verbatim. That is why there is random drop-out and learning rates.

I do not need to encode 1 + 1 = 2 in order to solve for 2. Thus it is the same for an LLM, only with an astronomical state space.

I do agree with you, however, that if you misrepresent what is happening sufficiently, people will indeed disagree with him- whether or not you're right or wrong (it's wrong, in case you were in doubt)

Slashdot Top Deals

The rate at which a disease spreads through a corn field is a precise measurement of the speed of blight.

Working...