Lying to you to give you that terrible restaurant recommendation. https://arxiv.org/pdf/2510.06105 is a white paper mathematically proving that LLMs will lie.
I have said this all along- most of AI is GIGO- Garbage in, Garbage out. LLMs were trained on the largest garbage producer in our society today, Web 2.0. Nothing was done to curate the input, so the output is garbage.
I don't often reveal my religion, but https://magisterium.com/ is an example of what LLMs look like when they HAVE curated training. This LLM is very limited. It can't answer any question that the Roman Catholic Church hasn't considered in the last 300 years or so. They're still adding documents to it carefully, but I asked it about a document published a mere 500 years ago and it wasn't in the database, but instead of making something up like most LLMs will do, it kindly responded that the document wasn't in the database. It also, unlike most AI, can produce bibliographies.