Finding a Place for AI in Medical Education

Artificial intelligence permeates our everyday lives in countless ways.  It only makes sense for it to also improve the way medical school students are trained, and how clinicians in practice carry out their everyday responsibilities.

By John Halamka, M.D., Diercks President, Mayo Clinic Platform and Paul Cerrato, MA, senior research analyst and communications specialist, Mayo Clinic Platform

The Association of American Medical College (AAMC) believes artificial intelligence should have a place in medical education and has published several key principles to help guide that position. As its website explains, it’s important to maintain a human-centered focus: “As AI technologies advance, human judgment remains essential in determining the appropriate use and implementation of these tools. Medical educators, staff, and learners must apply their critical thinking, creativity, and adaptability to effectively integrate AI into educational practices while maintaining a human-centered approach.”  As expected, the organization also encourages ethical and transparent use and equal access to AI. Equally important, the group states: “Investing in the education, training, and development of educators is essential to prepare them for the growing role of AI in medicine and to help them guide learners through this transformation. Creating a safe AI environment in which educators can explore its use is critical.”

Many researchers and developers have heeded the call for new, creative ways to incorporate AI in medical education. Yann Hicke, from Cornell University, along with investigators from several other schools, has developed a system that relies on generative AI to “enhance deliberate practice in medical education.” Called MedSimAI, it uses a large language model to create realistic interactive conversations with patients. The model, depicted in Figure 1, enables students to gain feedback using the Master Interview Rating Scale (MIRS), which has been used in the past to judge a clinician’s ability to provide effective, empathetic communication with patients. Among the 104 students who participated in the study, 28 responded to a survey about their impressions and generally viewed MedSimAI as “beneficial for repeated, realistic patient-history practice, with 78% highlighting focused history-taking and 62% highlighting question phrasing practice as its most useful function. While just over half (53%) cited the automated feedback as beneficial, a greater percentage emphasized access to repeated practice as the platform’s most prominent advantage.” (Source: Hicke  el al. MedSimAI: Simulation and Formative Feedback Generation to Enhance Deliberate Practice in Medical Education https://doi.org/10.48550/arXiv.2503.0579 )

Similarly, investigators from The University of Chicago and Northwestern University have created an LLM-powered medication education initiative called MediTools. It’s composed of three components: a dermatology case simulation tool, an AI-enhanced PubMed tool, and an enhanced Google News tool. The case simulation tool used real images taken from a public portal called DermNet and generated synthetic lab tested results with the help of ChatGPT-4o. The PubMed tool used PubMed APIs to assist with students’ queries and “used a custom function that we developed to help us extract metadata from the XML data like title, authors, year, abstract, and more.” The researchers used a service called Diffbot, which employed machine learning and computer vision to extract data from web pages that contained full text of articles retrieved from PubMed Central. The Google News tool took advantage of Google search API. Article summaries were created with the help of ChatGPT. Ten healthcare professionals were surveyed on the usability of MediTools, and said it could improve learning outcomes. However, the researchers were quick to point out that one of the weaknesses in their system is its reliance on ChatGPT with all its well-documented inaccuracies.

Of course, medical education includes a variety of related domains, including medical school, postgraduate education, and the education of physicians once they enter full time clinical practice. Mayo Clinic has been developing innovative ways to use AI in all these domains.* For instance, The Mayo Clinic Alix School of Medicine (MCASOM) has developed a tool aimed at enhancing the medical school application review process by using an advanced AI modeling framework. The AI model leverages a variety of textual features and generates summaries created for reviewers and thus far has achieved a greater than 80% performance rate. The MCASOM admissions team will explore the model’s efficacy and accuracy during the coming admissions cycle.

Mayo Clinic has also used an AI-based communication platform with its Arizona-based PGY-2 neurology residents, with the goal of improving residents’ ability to communicate effectively with stroke patients in the emergency setting. The pilot project used a platform to simulate a conversation with a patient and their family member presenting to the emergency department with weakness and language difficulties due to a stroke from left middle cerebral artery (MCA) large vessel occlusion. Residents were tasked with counseling the patient for acute interventions, including thrombolysis and thrombectomy. During this simulated patient counseling, residents were evaluated on the use of medical jargon, clarity of speech, active listening skills, and flow of conversation. Additional metrics that were assessed included the resident’s ability to accurately explain the risks of fatal intracranial hemorrhage, as well as review medical contraindications to thrombolysis, based on the American Heart Association and American Stroke Association (AHA/ASA) guidelines.

After residents completed the virtual encounter, they were given time to review their feedback on the simulation platform. Then, they completed an in-person simulation encounter with a standardized patient presenting with a right MCA syndrome. Residents were tasked with obtaining a focused history, examining the standardized patient, interpreting CT head and CT angiography, and consenting the standardized patient and acting spouse for thrombolysis and thrombectomy.

During the scenario, faculty assessed residents’ communication skills, sharing feedback that was similar to that reported by the AI tool. A five-question feedback survey was distributed at the end of the simulation, assessing the resident-perceived utility of the AI communication encounter and feedback in practicing patient counseling. In the final analysis, the majority of trainees found the feedback helpful and 80% reported they would be interested in using the AI tool again for future communication practice.

In the past, critics have often pointed out that healthcare has been behind other industries in adopting new technology. The latest research suggests we are catching up.

Footnote

*We would like to thank the many educators, researchers, and clinicians who contributed directly and indirectly to the development of this column. Megha Tollefson, MD, Imon Banerjee, PhD, Avisha Das, PhD, Aaron Pendl, MS, Dan Thompson, JD, MA, Mar Sayo, MBA, Zach Huston, MBA, Lisa Jore, MPA, PMP, Aimen Vanood, MD, David Lanman, BS, Varun Puri, BA, Matthew Hoerth, MD, Shiri Etzioni, MD