He's not nice, but he's also not wrong. You have some very odd ideas about what LLMs do.
LLMs absolutely, without question, do not learn the way you seem to think they do. They do not learn from having conversations. They do not learn by being presented with text in a prompt, though if your experience is limited to chatbots could be forgiven for mistakenly thinking that was the case. Neural networks are not artificial brains. They have no mechanism by which they can 'learn by experience'. They 'learn' by having an external program modify their weights in response to the the difference between their output and the expected output for a given input.
It might also interest you to know than the model itself is completely deterministic. Given an input, it will always produce the same output. The trick is that the model doesn't actually produce a next token, but a list of probabilities for the next token. The actual token is selected probabilistically, which is why you'll get different responses despite the model being completely deterministic. The model retains no internal state, so you could pass the partial output to a completely different model and it wouldn't matter.
I vividly remember a newspaper article that said Ai performed better if you asked it to think things through and work it out step by step.
LLMs do not and can not reason, including so-called 'reasoning' models. The reason output improves when giving a 'step by step' response is because you end up with more relevant text in context. It really is that simple. Remember that each token is produced essentially in isolation. The model doesn't work out a solution first and carefully craft a response, it produces tokens one at a time, without retaining any internal state between them. Imagine a few hundred people writing a response where each person only sees the prompt and partial output on their turn and they can only suggest a few potential next words and their rank, the actual next word selected probabilistically. LLMs work a bit like that, but without the benefit of understanding.
I think LLMs resemble the phonoligical loop a bit.
I assure you that they do not. Not even a little bit.
Pretty sure at some point self awareness is needed to stabilize the output.
You probably realize by now that this is just silly nonsense.
The bloody thing hallucinates for Christ's sake!
That's a very misleading term. The model isn't on mushrooms. (Remember that the model proper is completely deterministic.) A so-called 'hallucination' in an LLM's output just means that the output is factually incorrect. As LLMs do not operate on facts and concepts but on statistical relationships between tokens, there is no operational difference between a 'correct' response and a 'hallucination'. Both kinds of output are produced the same way, by the same process. A 'hallucination' isn't the model malfunctioning, but an entirely expected result of the model operating correctly.