Forgot your password?
typodupeerror

Comment Re:Everything is now (Score 0) 56

The irony is that this reveals you to be the "old internet user" who's used to "people on the internet" being different from "people I know in real life".

For younger internet users, there is little difference. Internet is where they interact with schoolmates, work colleagues, friends, teammates, etc. They see people like you and I as "these weird boomers who think internet isn't full of real people".

Meanwhile interest based groups are reasonably localized. I know something like 75-80% of people in most of those, and have met them face to face at least once. Same goes for those massive snapchat/whasapp/discord/etc groups. While some are international, for most of the young people these have in fact replaced bars, gentlemen's clubs, book reading circles, bowling alleys, etc social gatherings. They're the normal place where they socialize with friends and acquaintances now. And these old ones are slowly withering and dying as youngsters don't see a reason to go to a bar to meet up with friends any more.

Comment Re: all written from scratch? (Score 1) 76

You're an idiot.

If you anthropomorphize the term "learn", then you're correct. They learn nothing.
But then again, at that point, a heuristic spam filter learns nothing either, and yet we have called that learning for 30 years.
So, accepting that this level of flexibility is allowed for the word "learn", we can accept that the backpropagation is, in fact, learning.

Further, "learning" is the exact fucking opposite of novelty. It pains me that people as stupid as you have the boldness to share your half-baked opinions.

Comment Re: all written from scratch? (Score 1) 76

Absolutely incorrect.
It is a statutory exception to copyright.
Where my fair use rights intersect with your copyrights, the rights granted by your copyright are null and void- and that's clearly written in the statute.

That part that is not clear, of course, is whether or not my opinion about whether something is fair use or not, is actually fair use- and from that perspective, you can say it's a defense, since I'll have to make that argument into a defense if a prosecutor doesn't buy it.

Comment Re: all written from scratch? (Score 1) 76

Agreed absolutely. However, as mentioned by me, if as described by the GP was their incorrect reduction of what an LLM does, then that question is absolutely still in flight, because Judges aren't morons who can be talked into seeing waterboarding as "essentially washing your face".

Comment Re:Everything is now (Score -1) 56

Small "we focus on interest X, we retain with comment sections and we monetize through ads" websites got killed a decade ago by facebook that centralized that model to their site. If you want to do basket weaving or local HEMA re-enactment group or anything in between in your area, your state, your nation, etc, there's a facebook group for that now. That is vastly superior in connecting people and generating good articles that aforementioned websites ever were, which is why facebook won.

Then reddit killed the actual forums in the exact same way. It's both better and more convenient to have a centralized depository for everything, with common interface and rule set from perspective of 95% of users (I pulled that number out of my ass, but you know exactly what I mean).

What's left is mostly various wikis, and similar "we provide information" websites. Those are getting hammered by AI being able to answer faster and better than having to go through a wiki for a few minutes for that specific interest you're looking for. This is notably why google is losing profitability. It's bread and butter was monetizing the search that found those sites, and then monetizing their ad stream through google ads that were common on such sites. Today, google's AI search is better at answering those questions (not to mention just asking questions of other popular models), but on top of that if your AI answer to the question being asked is on point and answers the question user is searching the answer for... you don't get to monetize it nearly as well.

But the hilarity is that internet is more alive than ever. It has more actual users than ever, as older generations who don't use it are dying out, while newer generations basically live their entire lives on the internet the way older ones couldn't even imagine. But with people being more and more able to specifically select for their more and more specialized and narrow interests, those of us used to much broader communities with broader interests are left with a feeling that everyone is gone.

Meanwhile every football guy and every party girl is in fifty snapchat/whatsapp/discord/etc groups that have to be muted because otherwise the phone wouldn't stop going off with notifications from other people in those groups talking.

Comment Re: all written from scratch? (Score 0) 76

but your fair use doesn't include publishing it.

Again, not necessarily.
Every country I'm aware of contains fair use provisions for transformative use. It would be strange if they didn't.

I answered with "not necessarily" because it really does have a lot more to do with the physical act of "taking your image and editing it with photoshop", not that that accurately describes how an LLM works at all.

This isn't to defend the pro-LLM argument by any means, but it is the argument, and it has succeeded in some courts, and failed in others.

Comment Re: all written from scratch? (Score 2, Informative) 76

Cool, and my grandma's goal could have very well been to win the Tour De France, yet it still didn't make her any more bicycle-like.

Your analogy is that your grandma wanting to achieve something unachievable for her is comparable to LLM pretraining specifically trying *not* to memorize token sequences?
Not very intelligent, are you?

Do you have any proof of that?

Do I have proof of that?
It's self-fucking-evident. Do you think a calculator decodes an unwrapped ALU output to answer your inputs?
But guess what- it could.

Because vast majority of people definitely see various forms of "1 + 1 = 2" repeat many times before they get to the level where they can derive that formula themselves, and I'd argue it's fairly easy to prove that vast majority of people actually remember that "1 + 1 = 2" and don't derive it in practice.

Strawman. Probably unintentional on your part given your demonstrated inability to hold a train of thought, but:
I asserted that I did not need to, you answered with "well I bet most people do."
If the simplified example was confusing, replace 1 + 1 = 2 with 44321 * 454531 = 20145268451. Yes, an LLM can solve that, and no, it has never seen it in its training data.

As far as I know, we have seen LLMs generally tending towards being more reliable at repeating things that are already in the data than they are at inventing novel things

LLMs are terrible repeaters, except for very contrived situations.
The fact that their knowledge base is centered around the exact data they have ingested is not a mystery. That being said, models generalize- and that is the goal of pretraining- for them to generalize. Repating is an error that you want to avoid, because a model that repeats is a dumb model.
We don't want the model to know that Paris comes after "What is the capital of France?". We want it to have encoded Paris as a thing called a capital of a place that is called france that is a country in europe. And they do.

and as far as I know, there's been zero LLMs that have written operating systems "from scratch" that haven't had multiple complete operating systems in their training data.

It would indeed be very unlikely that an LLM that had not seen examples of an operating system could ever generate one. That doesn't mean it reproduced what it learned. It means it learned the shape of an operating system from its data.
Again, I do not need to encode 44321 * 454531 = 20145268451 to solve for 44321 * 454531.

I've been training neural nets since well before the initial Transformers paper.

Perhaps you have, but in the same way an LLM would- you had no fucking idea what you were doing ;)

You vomiting a couple [overfitting mitigation methods] isn't impressive to me.

I suspect nothing impresses a bullshit artist such as yourself- it's part of your persona to hide that.

We've known for a long time that some information is more useful when blurred, and often easier to memorize when less specific. Yes, of fucking course, we're aiming for generalization in AI models.

And your regurgitation of things you don't understand isn't impressive to me.

But I you haven't shown anything to prove that this particular LLM isn't just sticking together pieces of kernels it has seen. Nobody calls a collage "from scratch".

It's quite easy to prove that this doesn't contain a single line of linux within it. Nobody said it doesn't contain the semantic concepts that it learned from linux- that was, in fact, the entire point you dim-witted poser.
The models knows what goes into a scheduler. It can reproduce a scheduler with any kind of variation you need within the shapes of other concepts it has learned. It does not need the code it took to train its weights to understand that shape.

Nobody calls a collage "from scratch".

And nobody calls someone else's code a collage, though by your definition, it most certainly is.

Comment Re:It's an interesting metric (Score 1) 45

I think that is ultimately what the industry is trying to figure out right now.
It's certainly cheaper to use tokens in a subsidized model, but the subsidized model is bullshit- so there's that. They can't keep selling the tokens at a loss forever.
At raw API prices, a human is flatly cheaper for this kind of work, and likely more reliable.
Of course the flip side of that, is that the inference itself isn't what's expensive, it's the model arms race, which likely isn't sustainable in any scenario.

Slashdot Top Deals

"There... I've run rings 'round you logically" -- Monty Python's Flying Circus

Working...