Comment Re:PE Vultures are at it again (Score 2) 112
Same thing with low-resource languages. The more problem is the more obscure domain/language, the less actual training corpus, the more nonsense output from the robot. Eg ask it to write reasonably complex win32 and posix command tool for same problem, with the only differences there being the semantic differences between the systems. It will overall hallucinate more nonsense for win32.
The robot is very good at what it knows - especially at the center of distribution (not too much extrapolation). And it knows what's common. The more you touch edge of the domain, the accuracy nosedives.