generally they fail at the balanced parenthesis problem
LLMs are way better than humans at ensuring that code parentheses are balanced, e.g. when writing code; LLM code is much more likely to compile / run on the first time than human code. There are literal parenthesis-counting circuits that emerge. That they're not perfect at it doesn't make them not be better than us at it. Humans tend to start to lose track after 3-4 levels of parentheses. LLMs have long since significantly outperformed us in nested grammatical tasks, so long as the level of training to the human and AI are at least remotely comparable.
Here is, internally, how models handle counting, despite not being able to see what they look at.
Also, your article is deeply dated, in terms of (A) models, (B) underlying articles, and (C) descriptions of how things work. Strange for something written in 2023; it reads like something written in 2020-2021. Just to give an example of what I mean, let's just grab a few paragraphs from the middle at random:
It operates in three basic stages. First, it takes the sequence of tokens that corresponds to the text so far, and finds an embedding (i.e. an array of numbers) that represents these. Then it operates on this embedding—in a “standard neural net way”, with values “rippling through” successive layers in a network—to produce a new embedding (i.e. a new array of numbers). It then takes the last part of this array and generates from it an array of about 50,000 values that turn into probabilities for different possible next tokens. (And, yes, it so happens that there are about the same number of tokens used as there are common words in English, though only about 3000 of the tokens are whole words, and the rest are fragments.)
A critical point is that every part of this pipeline is implemented by a neural network, whose weights are determined by end-to-end training of the network. In other words, in effect nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data.
There are, however, plenty of details in the way the architecture is set up—reflecting all sorts of experience and neural net lore. And—even though this is definitely going into the weeds—I think it’s useful to talk about some of those details, not least to get a sense of just what goes into building something like ChatGPT.
First comes the embedding module. Here’s a schematic Wolfram Language representation for it for GPT-2:
The input is a vector of n tokens (represented as in the previous section by integers from 1 to about 50,000). Each of these tokens is converted (by a single-layer neural net) into an embedding vector (of length 768 for GPT-2 and 12,288 for ChatGPT’s GPT-3). Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens, and from these integers creates another embedding vector. And finally the embedding vectors from the token value and the token position are added together—to produce the final sequence of embedding vectors from the embedding module.
Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this. It’s just that various different things have been tried, and this is one that seems to work. And it’s part of the lore of neural nets that—in some sense—so long as the setup one has is “roughly right” it’s usually possible to home in on details just by doing sufficient training, without ever really needing to “understand at an engineering level” quite how the neural net has ended up configuring itself.
Unless this is awkwardly worded, it does not "find an embedding (i.e. an array of numbers) that represents [the input sequence of tokens]". Each token has its own embedding. And they don't need to be "found", they're just a lookup.
Values don't go “rippling through successive layers" in a LLM. The core of a LLM is dozens to hundreds of individual DNNs (feed-forward networks) separated by attention blocks and add+norm. He entirely writes this out of the picture (at this point - later he reinserts them but does so incorrectly), but they are utterly critical to a LLM's ability to function.
"A critical point is that every part of this pipeline is implemented by a neural network" - No, it isn't.
"nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data" - No, it isn't. Creating your vocabulary for example is done with an algo like BPE or unigram. Not neural network based.
"Here’s a schematic Wolfram Language representation for it for GPT-2 .... 12,288 for ChatGPT's GPT-3" - I hope you understand how ancient these architectures are by now.
"Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens..." - to the point that they don't even have RoPE yet.
"Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this." - Uh, yes, there very much is. This isn't some sort of cargo-cult nonsense, it's a basic property of latent spaces. Information in a latent space (in most latent spaces) is encoded by directionality. You can merge multiple concepts into a latent space or adjust the properties of a given latent by summing them together, thus creating a vector that incorporates both directions.
Anyway, that's enough to show what I think of the quality of your reference, esp. in a modern setting. :P Moving on:
It's related to the clock problem
Multimodal input isn't as advanced as textual input, to be sure, but let's not pretend that the human brain isn't also readily tricked in various kinds of vision problems.
If you ask a neural network to recognize "A" then all As will be tokenized the same way
Sorry, that is, again, not how it works. Tokenization happens before the model ever touches the input. It's literally the first step. It has no say over how it's done. It can't say "tokenize this differently". Even the tokens themselves don't make it to the LLM; they only point to latent positions.
