Comment Re:Thinking (Score 2) 19
The difference between a Senior Developer and a Junior Developer is that a Senior will admit when they have no clue about what you're asking.
Experts have no problem with saying "I don't know."
The reason we trust senior developers is that they have a proven track record, and also will tell you when something is new and they don't know it (although rare), and then they go and try to figure it out if they can. If they cant figure it out they will still tell you they cant fix it.
AI doesn't know whether its telling the truth or not. It doesn't know what it doesn't know. I just has training data that tells it which answer is likely correct.
Fundamentally it doesn't have the ability to know its own limitations, no training data can fix that. Judgement and Wisdom comes from experience, experience comes from the consequences of getting it wrong. AI faces no consequences for its answers as such it will never have real Judgement or Experience, it can only borrow the experience of others through training data.
For Humans lying about what you know generally only happens when you're in a dysfunctional job environment that punishes honesty or you lied to get the job you have and you're not qualified. Also not knowing or misunderstanding is very different than making shit up.
Part of the reason AI cant really bring work savings on anything non-trivial is if I need to sign off on something I really need to check and understand everything AI tells me, line by line (which in many cases takes as long or longer than writing it myself). If I'm not doing that I shouldn't be trusted to sign off on stuff. Bluntly this means AI agents running against production is an absolute no go (Just ask Lemkin: https://www.zdnet.com/article/... ). Automation code written by AI, verified and signed off by a human admin can be run against prod, but only after being tested against dev / test.
Humans are pretty good at recognizing when they are dealing with something new or unknown (or at least smart enough to not try random sh!t), where we suck is predictable repetitive stuff, and keeping large data sets in our brains.
Hallucinations are "training data gaps" (if we're nice) or at worst trying random answers from known training data when dealing with something new or uncertain, rather than say "I don't have a clue".
Bluntly I think in _some_ situations AI and humans can complement each other, when used responsibly and as a tool in a constrained solution space.
When you're ready to have AI sign off on your medical treatments and drug orders or any other matter of safety without a human in the loop let me know... (then we can either admit AI might be ready or commit you to the asylum).