Forgot your password?
typodupeerror

Comment Re:Not as impressive as it sounds ... (Score 1) 43

You actually have no way to know whether or not Astra "figured" it out, and the evidence is that it could have- if you had been bothered to read.
Was Qwen's model card in Astra's weights? Probably.
Can Astra also design networks *from scratch* to solve problems? Yes. Can it troubleshoot other problems using mechanistic interpretability? Yes.

So no, it's not like saying that. Your analogy is as dim-witted as you are.

Comment Re: all written from scratch? (Score 1) 86

Your emphasis is irrelevant.
The fact that the exemption is an affirmative defense does not change the meaning of the statute-
Your fair use rights, which derive from the first amendment, directly override copyright rights.
The fact that you have to assert them in court is meaningless.

My fair use can give me permission to use your copyrighted work.
Nothing about your copyrighted work can take away my fair use.

Ergo, and I quote:

My fair use overrides your copyright.

Comment Re:Not as impressive as it sounds ... (Score 1) 43

I think you're being overly sensitive, and it's causing a dip in your ability to think critically.

What's newsworthy, is that the effective reasonbliteration was quarter-backed by an LLM, including the interpretability portion, and the modifications.
It did this to make a functional improvement to a specific metric after it was identified as the reason the metric was struggling.

The headline: MIT Researchers used GPT 6 Astra to edit Qwen's Open Weights Directly Boosting Math Scores is accurate.
Your proposed headline is also accurate, but it leaves out what's noteworthy and the goal.

Comment Re:Misleading presentation (Score 1) 43

You can run the network and see which activations consistently pop up with high values relative to a control set. You can then go and down weight them which can suppress the behaviour, for example.

That's exactly how it's done. It's a pretty old process for trying to hack out censorship from the models.
It works, but only kinda.

Comment Re:correcting repetitive reasoning? (Score 2) 43

Weights are just scaling factors

Functionally incorrect. Roughly equivalent to saying, "Synapses are just chemical transmitters".

So why train then? If GPT-6 knows how an LLM's weights should be assigned, why are we not done?

Is this an actual question?
"Why do we practice for the game when we can just get some individual coaching on our 3-pointer?"

You really are a fucking idiot.

Comment Re: all written from scratch? (Score 1) 86

You're an idiot.

If you anthropomorphize the term "learn", then you're correct. They learn nothing.
But then again, at that point, a heuristic spam filter learns nothing either, and yet we have called that learning for 30 years.
So, accepting that this level of flexibility is allowed for the word "learn", we can accept that the backpropagation is, in fact, learning.

Further, "learning" is the exact fucking opposite of novelty. It pains me that people as stupid as you have the boldness to share your half-baked opinions.

Comment Re: all written from scratch? (Score 1) 86

Absolutely incorrect.
It is a statutory exception to copyright.
Where my fair use rights intersect with your copyrights, the rights granted by your copyright are null and void- and that's clearly written in the statute.

That part that is not clear, of course, is whether or not my opinion about whether something is fair use or not, is actually fair use- and from that perspective, you can say it's a defense, since I'll have to make that argument into a defense if a prosecutor doesn't buy it.

Comment Re: all written from scratch? (Score 2) 86

Agreed absolutely. However, as mentioned by me, if as described by the GP was their incorrect reduction of what an LLM does, then that question is absolutely still in flight, because Judges aren't morons who can be talked into seeing waterboarding as "essentially washing your face".

Comment Re: all written from scratch? (Score 0) 86

but your fair use doesn't include publishing it.

Again, not necessarily.
Every country I'm aware of contains fair use provisions for transformative use. It would be strange if they didn't.

I answered with "not necessarily" because it really does have a lot more to do with the physical act of "taking your image and editing it with photoshop", not that that accurately describes how an LLM works at all.

This isn't to defend the pro-LLM argument by any means, but it is the argument, and it has succeeded in some courts, and failed in others.

Comment Re: all written from scratch? (Score 2, Informative) 86

Cool, and my grandma's goal could have very well been to win the Tour De France, yet it still didn't make her any more bicycle-like.

Your analogy is that your grandma wanting to achieve something unachievable for her is comparable to LLM pretraining specifically trying *not* to memorize token sequences?
Not very intelligent, are you?

Do you have any proof of that?

Do I have proof of that?
It's self-fucking-evident. Do you think a calculator decodes an unwrapped ALU output to answer your inputs?
But guess what- it could.

Because vast majority of people definitely see various forms of "1 + 1 = 2" repeat many times before they get to the level where they can derive that formula themselves, and I'd argue it's fairly easy to prove that vast majority of people actually remember that "1 + 1 = 2" and don't derive it in practice.

Strawman. Probably unintentional on your part given your demonstrated inability to hold a train of thought, but:
I asserted that I did not need to, you answered with "well I bet most people do."
If the simplified example was confusing, replace 1 + 1 = 2 with 44321 * 454531 = 20145268451. Yes, an LLM can solve that, and no, it has never seen it in its training data.

As far as I know, we have seen LLMs generally tending towards being more reliable at repeating things that are already in the data than they are at inventing novel things

LLMs are terrible repeaters, except for very contrived situations.
The fact that their knowledge base is centered around the exact data they have ingested is not a mystery. That being said, models generalize- and that is the goal of pretraining- for them to generalize. Repating is an error that you want to avoid, because a model that repeats is a dumb model.
We don't want the model to know that Paris comes after "What is the capital of France?". We want it to have encoded Paris as a thing called a capital of a place that is called france that is a country in europe. And they do.

and as far as I know, there's been zero LLMs that have written operating systems "from scratch" that haven't had multiple complete operating systems in their training data.

It would indeed be very unlikely that an LLM that had not seen examples of an operating system could ever generate one. That doesn't mean it reproduced what it learned. It means it learned the shape of an operating system from its data.
Again, I do not need to encode 44321 * 454531 = 20145268451 to solve for 44321 * 454531.

I've been training neural nets since well before the initial Transformers paper.

Perhaps you have, but in the same way an LLM would- you had no fucking idea what you were doing ;)

You vomiting a couple [overfitting mitigation methods] isn't impressive to me.

I suspect nothing impresses a bullshit artist such as yourself- it's part of your persona to hide that.

We've known for a long time that some information is more useful when blurred, and often easier to memorize when less specific. Yes, of fucking course, we're aiming for generalization in AI models.

And your regurgitation of things you don't understand isn't impressive to me.

But I you haven't shown anything to prove that this particular LLM isn't just sticking together pieces of kernels it has seen. Nobody calls a collage "from scratch".

It's quite easy to prove that this doesn't contain a single line of linux within it. Nobody said it doesn't contain the semantic concepts that it learned from linux- that was, in fact, the entire point you dim-witted poser.
The models knows what goes into a scheduler. It can reproduce a scheduler with any kind of variation you need within the shapes of other concepts it has learned. It does not need the code it took to train its weights to understand that shape.

Nobody calls a collage "from scratch".

And nobody calls someone else's code a collage, though by your definition, it most certainly is.

Slashdot Top Deals

One good suit is worth a thousand resumes.

Working...