These autonomous digital tools can perform complex clinical and administrative tasks once considered the domain of humans. While they may ease our workload, we still need to be aware of the downside.
By John Halamka, M.D., M.S., Dwight and Dian Diercks President, Mayo Clinic Platform and Paul Cerrato, MA, senior research analyst and communications specialist, Mayo Clinic Platform
Large language models (LLMs) have had a profound impact on medicine in the last several years. But stand-alone LLMs have their limits. What distinguishes agents is that they act on outputs of LLMs (and multi-modal models generally). They don’t replace LLMs, rather they leverage them to automate actions.
Bernardo Collaco and his associates explain the difference this way, “AI agents distinguish themselves from previous AIs by adapting autonomously with minimal human intervention, and integrating effectively with specialized models, often operating within sophisticated multi-agent frameworks and invoking external tools. The ability of these agents to manage complex tasks collaboratively, offering a higher degree of operational autonomy and adaptability, is a hallmark of next-generation AI systems... Specifically, AI agents that exhibit the highest levels of decision-making, adaptability, and self-sufficiency are referred to as Agentic AIs.”
A recent Mayo Clinic study illustrates their expanding role as it applies to patient education. Yuexing Hao and associates have created a LLM agent that is linked to the hospital’s EHR system. Called MedEduChat, it provides personalized education to patients with prostate cancer. The app gives them plain English explanations of complex medical terms and helps them better understand their diagnosis and treatment regimens. A recent study published in Digital Medicine found that the content generated by MedEduChat was accurate, safe, and complete. It also reported a high usability score (UMUX = 83.7/100) and an increase in health confidence score from 9.9 to 13.9.
Mayo Clinic has also been developing a patient safety solution that relies on AI agents. To date, the Enhanced Triage Agent has been completed and is in production; the Pattern Discovery Agent will be completed shortly. The Patient Safety Vital Sign (PSVT) initiative has progressed from a strategic concept into a defined proof-of-concept (POC) program. The initiative aims to create a real-time, composite patient safety risk score that synthesizes multiple safety domains into a single actionable signal for frontline nursing teams. This solution aims to address the many preventable problems occurring among inpatients, including falls, healthcare-associated infections, pressure injuries, medication-related events, and clinical deterioration.
German investigators have done some of the most recent work in agentic AI. Dyke Ferber and associates developed an AI system called Medical Intelligence for Reasoning and Action, or MIRA. It can collect and analyze several sources of patient data, including medical history, physical exam, lab results, medications, imaging scans, and surgeries and is integrated with a hospital’s EHR system. And unlike stand-alone LLMs, which might make a recommendation on one aspect of the patient’s case but don’t take any action, MIRA was able to act by itself. Their report found that “In simulations on real patient cases spanning multiple diagnoses, MIRA outperformed physicians in diagnostic accuracy and made guideline-concordant, medication-safe and appropriate admission decisions.”
An independent AI agent called AMIE (Articulate Medical Intelligence Explorer) has likewise leveraged the features of AI Agents to improve patient care. It’s comprised of two agents, one that carries on conversations with a patient, and a second agent that is using slow thinking to understand the context of the patient encounter. This enables AMIE to perform “multi-visit clinical management and dialogue.” Using Gemini’s capabilities, it combines “in-context retrieval with structured reasoning to align its output with up-to-date clinical practice guidelines and drug formularies.” Lievin et al. state that the AI system was just as effective as primary care physicians in reasoning and finding precise treatments.
While the evidence shows that AI agents can improve diagnostic accuracy and workflow efficiency, there are still problems to contend with. Most studies that use these digital tools have not involved actual patients. Equally important, they turn LLM outputs into actions, so they can magnify LLM deficiencies (such as hallucinations, errors, etc).
In addition, agents that string together the outputs of multiple LLMs can compound these problems and make it harder to disentangle when things go wrong. They are also prone to automation bias because people assume that the machine is right and lower their level of diligence and scrutiny. And finally, current workflows are complex and designed for quality and safety with many interdependencies. Introducing agents disrupts those workflows and without rigorous attention to change management and training, they could actually increase administrative burden and risk of harm.
We can’t stop the AI bus, and so far, the benefits of AI agents seem to outweigh their disadvantages. But we need to be cognizant of the pros and cons.
Share this:
