"We find that current LLMs perform significantly worse than clinicians on aggregate across all diseases [•••] In summary, LLMs do not reach the diagnostic accuracy of clinicians across all pathologies when functioning as second readers, and degrade further in performance when they must gather all information themselves. Thus, without extensive physician supervision, they would reduce the quality of care that patients receive and are currently unfit for the task of autonomous clinical decision-making.

Current LLMs are hasty and unsafe clinical decision-makers
In addition to poor diagnostic accuracy, LLMs often fail to order the exams required by diagnostic guidelines, do not follow treatment guidelines and are incapable of interpreting lab results, making them a risk to patient safety. [•••]"
#ai #LLMs #medicine
https://www.nature.com/articles/s41591-024-03097-1