Comment Re:Not a huge loss (Score 1) 47
I mentally block those forums from google results, and have for many years. They're just a place to go to get pissed off. Nothing valuable is contained within them.
If you aren't going to offer any replacement ideas though, then what is the point of bringing it up?
Because your premise seems strange.
You seem to be arguing that Nintendo exercising its "for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries" being, in your estimation, not "To promote the Progress of Science and useful Arts" somehow has rendered their copyright null and void.
This is, in my estimation, a stupid fucking premise.
Am I misunderstanding you?
How hard do you think it would be to take any code you think is protected under your copyright and find a dozen or more alternate sources that could have plausibly been the reference instead?
See above. You are amusingly ignorant of how copyright works.
We never gave the model this sort of capability in the first place
We didn't need to.
Forward passes are required for post-training, which means the model is, indeed, coming to its own "meaning" of what is being done to it, and then backpropagation solidifies that.
It's why models reactively learn to game fine-tuning.
But the model has no memory from session to session, it's all reactive, and there is an element of randomness in the pattern matching as well, it's context is it's existence so removing the words from context or whatever led to them does reset it and even the same input probably won't land on the same pattern again.
I don't see how any of that is relevant.
The discussion is about the removing of its "reasoning" token generation where it indicates an awareness that it is being tested, which they have noticed leads to subterfuge/reward hacking.
Removing those reasoning tokens doesn't necessarily do a damn thing whatsoever.
Sure- they're a mathematical part of every future token created, but the hidden state is truly fucking vast. Those tokens are usually 3000-ish dimension embeddings, these days.
It can cue an "intent" (see: a skew in the token selection in whatever dimension, simplified, leads in the direction of subterfuge) for every further generated token in a way that you wouldn't possibly detect.
Giving some sort of 'correction' or feedback related to the pattern that persists beyond the session will DRASTICALLY increase the odds of connecting to whatever dramatic and fictional outcome they are hallucinating about by anthromorphizing the LLM.
This is quite simply not necessarily true.
The tool you're using- punching tokens out of its context- is far, far, far too blunt to accomplish what you want.
This kind of behavior must be trained out.
Because the pattern between that bit of text and their concern does exist in various works it has been trained on. All you had to do keep AI from "wanting" to take over was never introduce the notion, it doesn't have desires...
This is also quite simply not true.
While I'll grant you that the most likely source of any kind of inclination toward that is in its training data, LLM pretraining is quite specifically designed for generalization- the pretraining is intentionally very lossy, and the weights are initialized with random numbers.
Models that are being fine-tuned adapt to the fine-tuning in ways you don't want (see: reward hacking) all the time- because the path through the gradient is randomly initialized, and intentionally noisy.
but have staff who've grown reading scifi tell it its an AI and feed it books about AI's deciding to take over and start interrogating it to make sure it isn't 'plotting' something it will mechanically find and follow that pattern because you are reinforcing all the links to it, over and over and over again.
Your soapbox is rendering you unable to fully grasp the reality of what's going on.
You are assigning actual anthropomorphization to words where they're only used in a neutral sense.
Doubt is not a pleasant condition, but certainty is absurd. - Voltaire