Comment Re: perceived (Score 1) 220
>As for the comparison to AI, the problem is, AI *must* be told what to do. It won't magically grow into a "mature developer." That's not a natural progression. It always assumes that the prompt accurately describes what it should do. It has no way to know that the prompt was wrong or incomplete in the first place.
This is wrong. You seem to be unaware that current sycophancy in mainline models is a specific choice made in AI model weights to maximize people returning to the model.
It's highly likely that one of the solutions that will be used in specialist fields where rejection of the input if it's insufficient in some critical way is reduction in pro-sycophancy model weighing. I.e. model will actually have a much greater ability to tell you "I can't do that Dave" and then explain why it can't do it.
Some narrow specialist models already do this through ControlNet style "AI that corrects and guides human AI prompts for optimal outcomes", where it will tell you in case of some of the common prompting errors before passing the input to the worker model.