Yeah, that's a good example. Also "the actual thinking process as opposed to the correlational hallucinations that constitute all LLM 'thought'."

Two different sides of the same hubris.

A little tutorial if you know a little bit of Python:

https://numpy.org/numpy-tutori...

And a book with more detail if you're more into math:

https://www.deeplearningbook.o...

Everything else is pretty much scaling up and introducing some restrictions on the basic model.

You see it in the way you just used the word "you." Model "collapse" is a misnomer. It's drift, which you see in humans in literally everything they do, and you can demonstrate to yourself just by repeatedly generating random numbers and calculating the mean.

Bullshit. The fundamentals of modern AI models can be understood by anyone with a basic knowledge of algebra, which you should have picked up in junior high. They're piecewise linear approximations and use exactly the same equation as the linear regression you learned in high school or first year university. The more advanced stuff is hacky restrictions on that basic design to tone down the model's flexibility and make it easier to fit.

The reason it's hard to understand is because a) people who have no idea what they're talking about try and handwave their way through it; b) people explaining it want you to think it's really, really sophisticated and hard to understand, c) you're using "understand" in an unrealistic way or d) some combination of the above.

I think there is room to argue though, that distillation might violate copyright while training on large volumes of large material does not.

Nope. There is an argument about whether training a model on source material violates copyright or not. Current court decisions in the US say that the training does not, but you must have acquired the material legally in the first place.

Current court decisions in the US say that the raw output of a model is not copyrightable. Never mind that the people doing the distilling have paid for that output so even if it were copyrightable they'd own it.

The only thing happening is violation of terms of service which haven't been tested in court and would (hopefully) fail that test. Otherwise good luck with any software you use to produce anything, compilers included. And that's why these companies are directly lobbying the US government to do, uh, something, about it. They're trying to turn a bug in their business model into a geopolitical issue.

Yup. A better article about it:

https://www.quantamagazine.org...

The "new electrical demand" and "expected growth" are usually based on plans, connection requests, etc. A lot of it is speculative and isn't being built.

You don’t need kW of energy to store or host an image but you do to host AI.

Kilowatts is not a measure of energy. You do need kilowatts to serve many image requests per second. You don't need much energy to serve a single AI request. You do need kilowatts to serve many per second.

To put it in a car analogy, you've decided you don't like Fords so you're just going to say bad stuff about Fords that doesn't even really make sense.

I'm pretty sure it's our ability to assign properties to things based on zero evidence.

Or would that be un-American?

Apparently. There was a push to ban football around the beginning of the 1900s. It was properly brutal, with lots of people, mostly school kids, dying from injuries. Teddy Roosevelt though it was essential for building character though, and bullied many of the colleges into keeping their teams. He did compromise and push for important rule changes though.

https://www.history.com/articl...

Other recording devices do not constantly record on their own

Yeah, lots of them do. Your phone probably does, unless you have Siri/Google/whatever turned off. So do your Echos, Homes, Alexas, OnStars etc.

Anyway, I doubt any of the laws say anything about "constantly."

Here's California's:

A person who, intentionally and without the consent of all parties to a confidential communication, uses an electronic amplifying or recording device to eavesdrop upon or record the confidential communication, whether the communication is carried on among the parties in the presence of one another or by means of a telegraph, telephone, or other device, except a radio, shall be punished

Nope, no "constantly" at all. There is an "intentionally" and a "confidential" though.

Note that topology is often called "the study of holes."

It's quite funny watching people complain about datacentres by posting on social media. Especially when they use incredibly inefficient pictures of text instead of just text.

You've been watching too much Star Trek. Emotions are some of our most basic instincts. Overcoming them to become reasonably good token predictors is what separates us from lizards.

So can a regular phone. Or a tape recorder. Any recording device, in fact.

That's a possibility. One that doesn't obviously have fewer problems associated with it, is simpler, or better supported by evidence than the alternatives.