You're the one making the claim that modern frontier models still make this mistake (they don't), so how about you go to a frontier model, reproduce what you're claiming happens, and click the "share" button and post the link here?

Don't worry, I'll wait.

Your inability to reproduce the thing that you claim happens is duly noted.

You are speaking irrelevant nonsense. LLMs are trained in words

They are not. They are trained in tokens. Tokens do not align with word boundaries, and an arbitrary word can be tokenized in many different ways.

and they think in words

They do not. They don't even think in tokens. The process is: words are split to tokens, tokens point to an embedding position (latent space) while RoPE encodes a relative position, and all reasoning is done within latent space, which is not at all verbal (concepts are directions in latent space, and math is done on concepts, not words).

Also, a note: when spec'ing a generator, you need to know how much you're planning to use it vs. batteries. If it's only going to be used rarely, you prefer low mass, low volume, low cost, and low maintenance when unused (at the cost of low efficiency and higher maintenance in use), whereas if it's going to be used a lot, you prefer high efficiency and low maintenance cost in use, even if at the cost of higher mass, volume, cost, and maintenance when unused. In the former case, you'd prefer to allocate that extra mass, volume, and money into a larger battery pack.

That's why you don't use a tiny petrol generator? Diesel generator efficiencies are roughly:

Small backup generator (1-15kW): ~20-28%
Midsize backup generator (20-200kW): ~30-35%
Large industrial generator (200-2000kW): ~35-42%

Also, ironically this company's plan of the trailer providing a boost will actually make the tractor less efficient. ICE engines use "brake specific fuel consumption" (BSFC) graphs to plot their efficiencies across different RPMs and different torques. You can see an example for a small diesel engine here. Note that they require very high torque conditions and relatively high power conditions to be efficient. You can change the balance between torque and RPM within a given power band (blue) via gearing but gearing doesn't change what power band you're in. If you're in a low power band, you're fundamentally forced into inefficiency (note also that you're not going to be driving around at 1000 RPM just over a stall all the time).

Indeed, if you were forced into a low power band, you'd actually be better off with a series hybrid powertrain, as the engine can alternate between operating in an efficient powerband and shutting down. Of course, parallel hybrids are more efficient than series (albeit with added complexity and mass).

Unfortunately, the math doesn't work that way (even ignoring that a 400kWh battery is very small). Battery packs taper the closer you get to full, they're not a constant power all the way. Unless your battery pack can take 400kW at 80%, you're not charging that quickly.

Also, while 40 mins is fine in Europe (breaks: 45 minutes every 4,5 hours of driving... though using 70% of a 400kWh pack on a loaded class 8 truck going even at a slow 80kph will only take you 2 1/2h of driving in "average" conditions, so the truck's range is fundamentally undersized), the US is 30 minutes total break in 11 hours of driving, so ~6 hours on your first leg and ~5h on your second leg with 30 minutes to fill that 5h of driving. And US speed limits are usually faster for trucks than in the EU, so higher consumption. EU really needs 600kWh and >=600kW charging, while the US needs 800-1000kWh and >MW charging.

Note that in all of this we're assuming efficient-shaped trucks (Tesla Semi or the like), not your typical EU bricks, along with a well optimized powertrain and an efficient tyre config. If not, you need to increase those packs and charge powers further.

The military implications are obvious. Think Ukraine. If you suspect the enemy is trying to infiltrate on a dark night along several kilometers of frontline, you light up the scene while launching a bunch of low-cost FPV drones, and those infiltrators are about to have a bad day.

You *can* spot infiltrators in the dark with IR cameras, but it requires much more expensive drones and isn't usually as effective, hence the preference for night operations. Plus, there's IR camouflage, with varying degrees of success. But it usually makes you stand out like a sore thumb under illumination (you're basically wearing a tent).

This is a small prototype, not a full-scale mirror.

Also, ignoring that, pretending light brighter than the full moon is useless is... silly? For generating power, sure, but for illumination, it absolutely is not.

Can we add "surprise and delight" to that list of firing reasons?

It's simple addiction, based on a very low-entry, socially approved neural anesthesia.
Think Trainspotting writ small, played out in public where every time the family stops moving, Mom or Dad (or more often, both) pop out their phones to "just check some things".

Ah, but you may be mistaken about the point.
If you're talking about the simple physical reality it accomplishing the task of detection, well sure, it might be impossible. But if you recognize the point isn't to detect nukes but to gain nearly infinite streams of funding based on ok-perhaps-it's-implausible-but-hear-me-out -nukes-are-scary, well sir that certainly is within our reach.

This is akin to altering the English alphabet such that quantity and shape of symbols in the alphabet is different before teaching students English

No, it's not. It's akin to having secret numbers (that correlate neither with the spelling or meaning of the word) associated with each word, and the only way you can learn the secret numbers is if e.g. someone says "The secret number for Strawberry is '3'."

There are no other "clues" available to the LLM. It cannot see the property in question (spelling). It is a hidden "magic number" for all effects and purposes.

as there are daily videos of people doing this exact shit?

Literally go to a non-tiny AI right now and try it. Your pick. Try all the major ones - ChatGPT, Claude, Gemini, Grok, your pick. Then share me a link to the failure case you're claiming you'll get. Don't worry, I'll wait.

(Note: tiny: the "Google Answers" AI is a truly minuscule model, the sort of thing you could run on a cellphone with ample capacity to spare, and is only representative of minuscule models)

how about the guy that put 3 AIs together and told them to count to 100?

Sounds like what you actually have a complaint about is the (non-AI) voice interface they're linked to, because that's what screwed them. That entire premise doesn't work at all if you remove it and just do the test in text. Again: try it yourself.

generally they fail at the balanced parenthesis problem

LLMs are way better than humans at ensuring that code parentheses are balanced, e.g. when writing code; LLM code is much more likely to compile / run on the first time than human code. There are literal parenthesis-counting circuits that emerge. That they're not perfect at it doesn't make them not be better than us at it. Humans tend to start to lose track after 3-4 levels of parentheses. LLMs have long since significantly outperformed us in nested grammatical tasks, so long as the level of training to the human and AI are at least remotely comparable.

Here is, internally, how models handle counting, despite not being able to see what they look at.

Also, your article is deeply dated, in terms of (A) models, (B) underlying articles, and (C) descriptions of how things work. Strange for something written in 2023; it reads like something written in 2020-2021. Just to give an example of what I mean, let's just grab a few paragraphs from the middle at random:

It operates in three basic stages. First, it takes the sequence of tokens that corresponds to the text so far, and finds an embedding (i.e. an array of numbers) that represents these. Then it operates on this embedding—in a “standard neural net way”, with values “rippling through” successive layers in a network—to produce a new embedding (i.e. a new array of numbers). It then takes the last part of this array and generates from it an array of about 50,000 values that turn into probabilities for different possible next tokens. (And, yes, it so happens that there are about the same number of tokens used as there are common words in English, though only about 3000 of the tokens are whole words, and the rest are fragments.)

A critical point is that every part of this pipeline is implemented by a neural network, whose weights are determined by end-to-end training of the network. In other words, in effect nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data.

There are, however, plenty of details in the way the architecture is set up—reflecting all sorts of experience and neural net lore. And—even though this is definitely going into the weeds—I think it’s useful to talk about some of those details, not least to get a sense of just what goes into building something like ChatGPT.

First comes the embedding module. Here’s a schematic Wolfram Language representation for it for GPT-2:

The input is a vector of n tokens (represented as in the previous section by integers from 1 to about 50,000). Each of these tokens is converted (by a single-layer neural net) into an embedding vector (of length 768 for GPT-2 and 12,288 for ChatGPT’s GPT-3). Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens, and from these integers creates another embedding vector. And finally the embedding vectors from the token value and the token position are added together—to produce the final sequence of embedding vectors from the embedding module.

Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this. It’s just that various different things have been tried, and this is one that seems to work. And it’s part of the lore of neural nets that—in some sense—so long as the setup one has is “roughly right” it’s usually possible to home in on details just by doing sufficient training, without ever really needing to “understand at an engineering level” quite how the neural net has ended up configuring itself.

Unless this is awkwardly worded, it does not "find an embedding (i.e. an array of numbers) that represents [the input sequence of tokens]". Each token has its own embedding. And they don't need to be "found", they're just a lookup.

Values don't go “rippling through successive layers" in a LLM. The core of a LLM is dozens to hundreds of individual DNNs (feed-forward networks) separated by attention blocks and add+norm. He entirely writes this out of the picture (at this point - later he reinserts them but does so incorrectly), but they are utterly critical to a LLM's ability to function.

"A critical point is that every part of this pipeline is implemented by a neural network" - No, it isn't.

"nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data" - No, it isn't. Creating your vocabulary for example is done with an algo like BPE or unigram. Not neural network based.

"Here’s a schematic Wolfram Language representation for it for GPT-2 .... 12,288 for ChatGPT's GPT-3" - I hope you understand how ancient these architectures are by now.

"Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens..." - to the point that they don't even have RoPE yet.

"Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this." - Uh, yes, there very much is. This isn't some sort of cargo-cult nonsense, it's a basic property of latent spaces. Information in a latent space (in most latent spaces) is encoded by directionality. You can merge multiple concepts into a latent space or adjust the properties of a given latent by summing them together, thus creating a vector that incorporates both directions.

Anyway, that's enough to show what I think of the quality of your reference, esp. in a modern setting. :P Moving on:

It's related to the clock problem

Multimodal input isn't as advanced as textual input, to be sure, but let's not pretend that the human brain isn't also readily tricked in various kinds of vision problems.

If you ask a neural network to recognize "A" then all As will be tokenized the same way

Sorry, that is, again, not how it works. Tokenization happens before the model ever touches the input. It's literally the first step. It has no say over how it's done. It can't say "tokenize this differently". Even the tokens themselves don't make it to the LLM; they only point to latent positions.

Actually, no, it doesn't. "S T R A W B E R R Y" can be tokenized e.g. as

"[start]", "S", " ", "T", " ", "R" ....

But more common it may look something like "[start]S ", "T R", " A W", " B ", "E R R", " Y[end]" or whatnot.

And anyway, anyone who's used a (non-tiny) model in the past 2 years or so knows that these sorts of issues aren't really a "thing" anymore. But the key point is that this is a test on an ability to count something that the LLM can't actually see (letters).