Comment Re:Are hallucinations even a problem? (Score 1) 56
> driving a car, industrial control, vibe coding are bottlenecked by hallucinations
No they're not, this isn't 2022 anymore. Let me reiterate, they're fairly trivial to spot and largely accounted for in serious autonomous systems. The issue is monkey paw property of the LLM - they're maliciously compliant by seeking shortcuts too greedily while skipping over subtler ontologies that aren't verbalized in corpus enough so as to differentiate into a concept ("carwash is typically somewhere you might want to drive to" wasn't solidified -> hilarious stupidity ensues). This sort of mistake isn't hallucination, but just plain stupidity of someone who's never quite understood what carwash is for, but only heard em mentioned few times in a book.
Yann LeCun does point at this, though I think he's too optimistic that DNNs could fill those gaps by having online learning experience in a closed loop. The way GPTs work is embarassing parallel SGD that you simply can't do when learning one speck of experience at a time. Would need to invent entire new optimization algorithm that scales just as SGD if not better, yet works sequentially.