more racist.

FTFY, even though I think AC was being intentional.

Closest I could find to the nub of the problem. Whether this is going to work would depend on very accurate weather AND climate modeling and I don't think we are anywhere close at this time. Due to butterfly effects, the prediction problem is probably unsolvable, so I think that means we would need a control system with extra capability that is constantly compensating for prior interventions. It reminds me of the fly-by-wire problem for aircraft with negative dynamic stability. Not even theoretically possible for a human to fly the thing if the computer burps.

I still don't know what AC was mumbling about, but the few posts on this story say all that needs to be said about the relevance of the EFF now.

Not worth much, but I do have a couple of takes on the topic. Main one in my Subject, but that's part of a general problem of broken economic models. The big donor may mean well, but the model only works until the donor starts calling bad shots, which is what always happens. But now I think even the "aligned business model" solution angle fails on the dimensional collapse problem, and we humans are NOT going to stop collapsing the dimensions. We're intrinsically simpleminded and will insist on more simplicity than reality involves.

Time for a "discussion" on building "trust" with Claude?

Yeah, same joke but AIs don't call for original humor. Still, I sadly thought the story had more potential for funny.

The military implications are obvious. Think Ukraine. If you suspect the enemy is trying to infiltrate on a dark night along several kilometers of frontline, you light up the scene while launching a bunch of low-cost FPV drones, and those infiltrators are about to have a bad day.

You *can* spot infiltrators in the dark with IR cameras, but it requires much more expensive drones and isn't usually as effective, hence the preference for night operations. Plus, there's IR camouflage, with varying degrees of success. But it usually makes you stand out like a sore thumb under illumination (you're basically wearing a tent).

This is a small prototype, not a full-scale mirror.

Also, ignoring that, pretending light brighter than the full moon is useless is... silly? For generating power, sure, but for illumination, it absolutely is not.

I'm guessing it was a rush to FP but I had quite a bit of trouble figuring out your intention from that short teaser. You should have given us a hint, perhaps by speculating about the bribery. However I do think Facebook is only "donating" small amounts of cash and most of the YOB's "eternal gratitude" is based on past services rendered. (In the YOB's case "eternal" means about two weeks. Until some other shiny object gets his attention.)

I know it seems intrinsically off topic to mention books on today's version of ye olde Slashdot, but I've been reading a bunch of Facebook histories lately, and serendipitously I'm just now finishing Careless People: A story of where I used to work: Power. Greed. Madness. by Sarah Wynn-Williams. Seems to be a classic case of one of those roads paved with good intentions. I think she got too personal in places, but I think there were a couple of key omissions that bother me more. Most important involved the intrinsic self-contradiction in Zuck's denial of responsibility. So on the one hand Zuck is claiming Facebook didn't cause all those bad things, up to an including lots of deaths, but on the other hand, why are the advertisers giving Facebook so much money? If Facebook doesn't work as advertised to the people buying the ads on Facebook, then this defense becomes "Meta is a total fraud taking money for nothing."

I think the "Facebook as a drug" aspect also deserved much more consideration. The tobacco industry as a bad example did get mentioned once, but the entire internet.org => Free Basics scam is a classic drug dealer story. Giving away free samples to get the suckers hooked and turn the blind eye when the addicts start doing bad stuff to pay for their next fix. I actually think that psychological addiction has become a bigger threat to society than chemical addiction. In general psychology is bogus hokum, but the applied psychologists have learned several things about selling soap, widget cleaning services, and politicians (with the dirtiest widgets of all). (Not sure I should go into the historical details because of the ageism thing, but I remember free cigarettes with my C-rats, though I never made the transition to full addiction. I was saved by a psychological block against paying to literally burn the product?)

Another fundamental problem is at an even higher level and I'm still unsure how to describe it. One hand is the "unite the world with better communications" thing but the other hand is the "divide and conquer" tactics of the worst abusers of Facebook and Instagram. But all in all I definitely haven't found any evidence for Facebook/Meta making the world better. In the book she argues that this part is much worse than I thought, but I'm too personally inclined to lean in that direction so I want to reserve judgment and read a few more books...

I feel like I need to add the personal disclaimer thing... I decided Facebook was a seductive waste of time some years ago. I had one of the old university-linked accounts so I was on for a long time, but my solution was to cut it to 5 minutes/day using timers. I think I was probably using Facebook at least half an hour per day before that, but the five minutes was enough to see the main news from actual friends and it also broke the urge to visit Facebook, so many days got skipped entirely. Never felt like I was missing out. Then in 2022 I was assassinated on Facebook. There was no option to find out why, only an option to submit to Zuck or be dead, and I'm not much inclined to submit. I did exercise the option to download "my" data from Facebook and spent a while looking through the small amount of recent stuff, but didn't find anything interesting. Leaving me with the hypothesis that it was some kind of politically motivated hit job?

Unclear to me what distinction you are trying to make. My suggestion for a tax-based solution approach involves focusing on the effects of monopolistic practices on profits, and it does not matter which form the monopolistic practice takes if profits can be linked to that practice. I'm not saying it's a trivial problem, but I think there are clear indicators of abusive situations. In this case, the critical metric is that customers have no freedom to choose where their apps are coming from.

It might be possible for Apple to argue this is a kind of natural monopoly over security concerns, but I'm saying that should not matter. The profits that come from "gatekeeper" status should still be taxed at a higher rate.

In solution terms, this example seems quite similar to what should have been done to Microsoft a LONG time ago. Imagine if Microsoft had been divided into several competing daughter companies, each starting with a copy of the OS source code and equal resources, both physical and human. The larger the company, the easier it is to achieve reasonably equal divisions, though some minor financial adjustments might be necessary to balance the books. After that, the daughter companies would have competed to provide the best versions of evolving OSes and the customers would have had real choices. If the Apple Store was split between several competing companies, it would work in a similar way. One of the new stores might focus on attracting more developers with lower fees, but I'd be inclined to use the store that focused on fewer apps of higher quality.

This is akin to altering the English alphabet such that quantity and shape of symbols in the alphabet is different before teaching students English

No, it's not. It's akin to having secret numbers (that correlate neither with the spelling or meaning of the word) associated with each word, and the only way you can learn the secret numbers is if e.g. someone says "The secret number for Strawberry is '3'."

There are no other "clues" available to the LLM. It cannot see the property in question (spelling). It is a hidden "magic number" for all effects and purposes.

as there are daily videos of people doing this exact shit?

Literally go to a non-tiny AI right now and try it. Your pick. Try all the major ones - ChatGPT, Claude, Gemini, Grok, your pick. Then share me a link to the failure case you're claiming you'll get. Don't worry, I'll wait.

(Note: tiny: the "Google Answers" AI is a truly minuscule model, the sort of thing you could run on a cellphone with ample capacity to spare, and is only representative of minuscule models)

how about the guy that put 3 AIs together and told them to count to 100?

Sounds like what you actually have a complaint about is the (non-AI) voice interface they're linked to, because that's what screwed them. That entire premise doesn't work at all if you remove it and just do the test in text. Again: try it yourself.

generally they fail at the balanced parenthesis problem

LLMs are way better than humans at ensuring that code parentheses are balanced, e.g. when writing code; LLM code is much more likely to compile / run on the first time than human code. There are literal parenthesis-counting circuits that emerge. That they're not perfect at it doesn't make them not be better than us at it. Humans tend to start to lose track after 3-4 levels of parentheses. LLMs have long since significantly outperformed us in nested grammatical tasks, so long as the level of training to the human and AI are at least remotely comparable.

Here is, internally, how models handle counting, despite not being able to see what they look at.

Also, your article is deeply dated, in terms of (A) models, (B) underlying articles, and (C) descriptions of how things work. Strange for something written in 2023; it reads like something written in 2020-2021. Just to give an example of what I mean, let's just grab a few paragraphs from the middle at random:

It operates in three basic stages. First, it takes the sequence of tokens that corresponds to the text so far, and finds an embedding (i.e. an array of numbers) that represents these. Then it operates on this embedding—in a “standard neural net way”, with values “rippling through” successive layers in a network—to produce a new embedding (i.e. a new array of numbers). It then takes the last part of this array and generates from it an array of about 50,000 values that turn into probabilities for different possible next tokens. (And, yes, it so happens that there are about the same number of tokens used as there are common words in English, though only about 3000 of the tokens are whole words, and the rest are fragments.)

A critical point is that every part of this pipeline is implemented by a neural network, whose weights are determined by end-to-end training of the network. In other words, in effect nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data.

There are, however, plenty of details in the way the architecture is set up—reflecting all sorts of experience and neural net lore. And—even though this is definitely going into the weeds—I think it’s useful to talk about some of those details, not least to get a sense of just what goes into building something like ChatGPT.

First comes the embedding module. Here’s a schematic Wolfram Language representation for it for GPT-2:

The input is a vector of n tokens (represented as in the previous section by integers from 1 to about 50,000). Each of these tokens is converted (by a single-layer neural net) into an embedding vector (of length 768 for GPT-2 and 12,288 for ChatGPT’s GPT-3). Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens, and from these integers creates another embedding vector. And finally the embedding vectors from the token value and the token position are added together—to produce the final sequence of embedding vectors from the embedding module.

Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this. It’s just that various different things have been tried, and this is one that seems to work. And it’s part of the lore of neural nets that—in some sense—so long as the setup one has is “roughly right” it’s usually possible to home in on details just by doing sufficient training, without ever really needing to “understand at an engineering level” quite how the neural net has ended up configuring itself.

Unless this is awkwardly worded, it does not "find an embedding (i.e. an array of numbers) that represents [the input sequence of tokens]". Each token has its own embedding. And they don't need to be "found", they're just a lookup.

Values don't go “rippling through successive layers" in a LLM. The core of a LLM is dozens to hundreds of individual DNNs (feed-forward networks) separated by attention blocks and add+norm. He entirely writes this out of the picture (at this point - later he reinserts them but does so incorrectly), but they are utterly critical to a LLM's ability to function.

"A critical point is that every part of this pipeline is implemented by a neural network" - No, it isn't.

"nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data" - No, it isn't. Creating your vocabulary for example is done with an algo like BPE or unigram. Not neural network based.

"Here’s a schematic Wolfram Language representation for it for GPT-2 .... 12,288 for ChatGPT's GPT-3" - I hope you understand how ancient these architectures are by now.

"Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens..." - to the point that they don't even have RoPE yet.

"Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this." - Uh, yes, there very much is. This isn't some sort of cargo-cult nonsense, it's a basic property of latent spaces. Information in a latent space (in most latent spaces) is encoded by directionality. You can merge multiple concepts into a latent space or adjust the properties of a given latent by summing them together, thus creating a vector that incorporates both directions.

Anyway, that's enough to show what I think of the quality of your reference, esp. in a modern setting. :P Moving on:

It's related to the clock problem

Multimodal input isn't as advanced as textual input, to be sure, but let's not pretend that the human brain isn't also readily tricked in various kinds of vision problems.

If you ask a neural network to recognize "A" then all As will be tokenized the same way

Sorry, that is, again, not how it works. Tokenization happens before the model ever touches the input. It's literally the first step. It has no say over how it's done. It can't say "tokenize this differently". Even the tokens themselves don't make it to the LLM; they only point to latent positions.

Actually, no, it doesn't. "S T R A W B E R R Y" can be tokenized e.g. as

"[start]", "S", " ", "T", " ", "R" ....

But more common it may look something like "[start]S ", "T R", " A W", " B ", "E R R", " Y[end]" or whatnot.

And anyway, anyone who's used a (non-tiny) model in the past 2 years or so knows that these sorts of issues aren't really a "thing" anymore. But the key point is that this is a test on an ability to count something that the LLM can't actually see (letters).

You have it backwards. They see words not tokens or letters.

Sorry, but no. Inputs are tokenized before being fed to the model.

Do you think LLMs don't know how words are spelled due to composition of token dictionaries

Do you think trainers, by habit, put their specific model's token dictionaries into the training data - let alone repeat it often enough in numerous different forms so as to promote memorization?

My personal hot take is, if you want to leach from the commons, you have to give back in some meaningful way.

Open source: great, go ahead, no issues - the act of open sourcing is giving back.
Closed source: what are you planning to give back in exchange?

For the 8 trillionth time, LLMs are "blind". They don't "see" words, only tokens. The only way they can know how those tokens are spelled is to memorize the spelling of every one of them. They can't just "look" at them and see how many Rs are in a given token (not without writing or calling a tool - something that they're actually very good at)

Also, your understanding of LLMs seems to be rooted in like 2022.

Also they are literally probing how LLMs make decisions. It's been well understood for a long time now. You can directly alter their thoughts and/or logic to lead them to alternative decisions. FYI, they work by chains of fuzzy logical inference.