Diagnostic audits, improved clinical reasoning skills, and AI-driven models all hold promise.
By John Halamka, M.D., M.S., Dwight and Dian Diercks President, Mayo Clinic Platform and Paul Cerrato, MA, senior research analyst and communications specialist, Mayo Clinic Platform
Although much attention has been given to patient safety in the professional press over the years, relatively little attention has focused on one of the most important aspects of patient safety: diagnostic errors. A 2015 report from the National Academy of Medicine pointed out that about 5% of adult outpatients in the United States experience a diagnostic error annually. The same report found that diagnostic mishaps contribute to about 1 out of 10 patient deaths, cause as much as 17% of hospital adverse effects, and affect approximately 12 million adult outpatients a year, which translates into 1 out of 20 Americans. About half of these errors may be harmful, according to a 2014 report from Singh et al. Among the 850,000 patients who died in U.S. hospitals annually, about 71,400 of these deaths included a major diagnosis that had not been detected.
Unfortunately, these statistics, which were reported in Reinventing Clinical Decision Support, have not improved recently. A 2024 report estimated “total serious harms annually in the USA to be 795,000 (plausible range 598,000–1,023,000).” Similarly, a 2023 analysis in JAMA Internal Medicine, concluded among more than 2,400 patient records, “missed or delayed diagnosis took place in 23%, with 17% of these errors causing temporary or permanent harm to patients.” Put another way, almost 1 out of every 4 patients are likely to be misdiagnosed.
The fact that the U.S. spends almost $5.3 trillion a year on healthcare begs the question: What causes this national epidemic? The list is long. Patients don’t always cooperate with their healthcare provider when asked to provide a detailed medical history, which makes it more difficult to locate the root cause of the problem. On the other hand, a clinician may be so busy or so distracted that they fail to answer the right questions or order the most relevant tests. It’s also possible for laboratories to make mistakes when testing specimens or fail to integrate the results in the patient’s record. Interoperability issues can also delay a diagnosis, especially if there are barriers that interfere with the transfer of data from one hospital or clinic to another. In other cases, cognitive errors may cause clinicians to arrive at faulty conclusions on what’s causing a patient’s condition.
While diagnostic errors are never going to be completely eliminated, several interlocking steps can make a significant dent on the problem. They include 1) training students and clinicians to improve their clinical reasoning skills so that they don’t fall victim to common mistakes like premature closure and automation bias; 2) systematic diagnostic audits; 3) improved workflow; and 4) cultural reform.
By one estimate, there are at least 24 cognitive errors that clinicians can fall victim to, including anchoring, availability bias, premature closure, attribution error, overconfidence bias, omission bias, framing effect, and representativeness. We’ve discussed several of these mistakes in previous publications. This list needs to be expanded to include AI-related cognitive errors, including automation bias, the tendency of some clinicians to rely too heavily on automated systems and AI models.
The accompanying figure illustrates several of the potential solutions to the diagnostic error problem, including the value of diagnostic audits and systemic surveillance. Abimanyi-Ochom et al’s systematic review found “There is some evidence that trigger algorithms, including computer based and alert systems, may reduce delayed diagnosis and improve diagnostic accuracy. In trauma settings, strategies such as additional patient review (e.g. trauma teams) reduced missed diagnosis, and in radiology departments, review strategies such as team meetings and error documentation may reduce diagnostic error rates over time.” Similarly, Singh and associates conducted a retrospective analysis of primary care visit records using EHR data to determine if certain triggers might suggest a diagnostic error. Possible triggers included an unexpected hospitalization with 14 days of the visit and an unscheduled follow up visit within 14 days. Their study suggested that EHR surveillance may help providers gain new insights into what’s causing diagnostic errors.
Of course, the “elephant in the room” is whether machine learning and large language models will have a positive impact on diagnostic errors. The research is mixed. Investigators from Boston Medical Center and Harvard Medical School examined the diagnostic capabilities of advanced LLMs, including Claude, ChatGPT, and Gemini. Their conclusion: “Advanced LLMs showed high diagnostic accuracy (>90%) in common scenarios, with Claude 3.7 achieving perfect accuracy (100%) in certain conditions. In complex cases, Claude 3.7 achieved the highest accuracy (83.3%) at the final diagnostic stage, significantly outperforming smaller models. Smaller models notably performed well in common scenarios, matching the performance of larger models.”
On the other hand, when an international team curated a dataset that included 2,400 real patient cases, and 4 common abdominal disorders, they found that LLMs did not accurately diagnose patients across all pathologies when compared to physicians.
It is unrealistic to imagine that diagnostic errors will be eliminated anytime in the near future; nonetheless several of the strategies we’ve discussed are worth closer examination. We owe it to our patients to get a better handle on an epidemic that is costly lives.
Singh H et al. The global burden of diagnostic errors in primary care. BMJ Qual. Saf. 2016 Aug 16;26(6):484–494 https://pmc.ncbi.nlm.nih.gov/articles/PMC5502242/
Share this:
