Performance of a LLM on the reasoning tasks of a physician

sinij writes:

In all experiments, the LLM outperformed physician baselines and displayed continued improvement from prior generations of AI clinical decision support. Our study suggests that LLMs have eclipsed most benchmarks of clinical reasoning, motivating the urgent need for prospective trials.

The future of healthcare looking more bleak as there is a push to automate diagnosis. While LLMs are very capable in some areas, notable weakness is relevance realization. That is, humans are by far more capable of developing heuristic models on how to restrict search space and focus on relevant information. Medical diagnosis is one such area. I fear misdiagnosis of rare conditions going to become commonplace as LLMs are widely adopted in the medical space.