Comment Re:So something I don't think anyone is asking (Score 1) 52
Why do AIs keep putting fake citations into cases they generate? An AI just regurgitates what it finds in its data set after all...
No, they actually don't. What they do is regurgitate the sequence of tokens most likely to follow the current token based on a model generated from the training data. They certainly can regurgitate the exact contents of the training data set, but they can equally make up strings based on the model weights. That's what makes their output so pernicious: since it's what would likely follow, we see it as plausible and don't immediately flag it as something we need to check on. The only way to find the problems then is to check everything regardless of how plausible it appears to be.