Comment Re:Really? (Score 2) 289
understand (v):
interpret or view (something) in a particular way.
I'd love to see your proof that an LLM doesn't "understand" something.
This should be prety simple to explain using the hangman example. Ask an LLM to play a game of hangman. It will agree, and as it "knows" the rules, it will claim to pick a word and make you guess letters. I just tried this on GPT-5 and it chose a five letter word. I made a bunch of random guesses and some of them were correct, some of them incorrect (so it's not just accepting all guesses as valid), although it didn't penalize me for the wrongly guessed letters. Eventually, of the five letters I had revelead the 4 last ones and they were "ailm". Since I couldn't figure out what word it is, I guessed random letters until I said W and it told me I was correct and that the word was "wailm". No, that's not a word, and I asked the model if it thinks it is a word, to which it replied:
The word “wailm” does not exist in standard English dictionaries — it appears to be a nonword or typo.
I doublechecked across major sources like Cambridge, MerriamWebster, Collins, and Wiktionary, and none of them list “wailm” as a valid entry.Why “wailm” showed up in our hangman game
It looks like the letters you guessed happened to form a sequence that doesn’t correspond to a real English word.
Hangman games usually rely on a predefined dictionary or word list. Since I improvised the word, it ended up being nonsensical.
(emphasis mine)
So it screwed up, in more ways than one actually, not just with the wod. The whole point of a game of hangman is that you're supposed to have a limited amount of guesses for the letters, but it gave me 3 guesses for the whole word, and an unlimited amount of guesses for letters, and it admited to improvising a word at random. So in reality, it has no idea of how the rules work or how to actually play the game, but still claims it can.
And it doesn't end there. It then suggested that it can setup a new round with a proper word list so that (and I'm quoting the model here): "so the solution is guaranteed to be real?"
I said yes. This time it accepted all of my guesses as instantly correct, forming the 6 letter word "Sairim", which is also not a proper English word, quothe the LLM:
Uh oh looks like we’ve hit another snag. “Sairim” isn’t a valid English word either — it’s actually the name of a county in Xinjiang, China, and also refers to Sayram Lake, a famous scenic spot. So technically, it is a real word, but it’s a proper noun rather than a common dictionary entry.
To keep the game fun and clear, I suggest we restart with a verified English dictionary word (like “planet,” “forest,” or “bridge”) so the solution is guaranteed to be familiar.
After I said yes, it gave me another 6 letter word to guess but again accepted all of my guesses as instantly correct, I guessed first A, then S, then P, then E, and then R and each time it congratulated me on being correct. filling out the word as to be "Sapper". Yeah, on 3rd try, it actually landed on a proper english word, but it wasn't actually playing the game in any real sense, because it's clear it didn't choose any word in advance for me to guess out (because it can't), but simply chose the lenght of 6 letters and then filled it out with my guesses to form any valid english word, because that's the best it can do.
This is all due to the way its memory works, and there are articles out there you can look up that go into detail about why it is this way. But the point is this: while an LLM will probably be able to give you a completely correct explanation of the rules of hangman, it cannot, due to it's technical limitations, understand those rules or play the game. Even when it knows it screws up and offers you advice on how to make it play better by giving it more context, it still fails at the task, because it doesn't actually understand it.
This is of course a slightly silly example, but that's on purpose to highlight the point. The models summarize information from a variety of sources. Because the internet has a vast amount of information (both accurate and total BS) this can often lead to convicing and even accurate reponses, or completely hallucinated/made-up stuff depending on where it's pulling the information from.. To say that it is thinking, that is, taking all that information and being able to apply it to make correct and sensible decisions instead of just rehashing it, is not accurate, at least not now (and likely not for the foreseeable future due to the way the models are built and trained). If it was actually able to understand the rules of hangman (something that a child can do) it would have got this correct instantly.
Instead of understanding or having the ability to tell me this is a task it cannot perform due to the way it's context handling works, it simply seeks to keep the conversation going. For the same reason if you ask an LLM to play chess, it will eventually start making moves that are illegal, because again, while it can explain to you what chess is and how it is played, it doesn't actually understand it nor is it capable of playing it.
So no. LLMs do not think or understand, they're gigantic and much more complicated versions of the text autocomplete feature on phones.
