We Need an Interdisciplinary Approach to AI

AI-enabled algorithms can’t exist in an operational vacuum. To really benefit our patients, they need to be deployed with the help of an interprofessional team of experts and a set of “delivery science” principles that most developers are not aware of.

By Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform and John Halamka, M.D., president, Mayo Clinic Platform

As data scientists, physicians, and informaticists develop innovative digital tools to improve the diagnosis and treatment of disease, they sometimes overlook the fact that these algorithms need to “play well with others.” To be truly useful to clinicians and administrators in everyday medical practice, they must fit into their workflow routines, offer benefits for them to actually want to use them, reduce their workload, and improve patient outcomes. To accomplish this rather formidable to-do list will require an interdisciplinary approach that is often lacking in many health care systems.

Ron Li, Steven Asch, and Nigam Shah, with Stanford University, explain it this way: “In order to safely and effectively bring AI into use in healthcare, there needs to be a concerted effort around not just the creation, but also the delivery of AI. This AI “delivery science” will require a broader set of tools, such as design thinking, process improvement, and implementation science, as well as a broader definition of what AI will look like in practice, which includes not just machine learning models and their predictions, but also the new systems for care delivery that they enable.”

A closer look at one of the mortality prediction models that Stanford has developed illustrates how many challenges exist when an AI model is put into a real world setting. Li and his associates created an all-cause mortality prediction model that they hoped would improve palliative care services with better advance care planning. But instead of forcing the algorithm onto unwilling end users, the team used proven design process improvement methodology to look for ways in which the model might break down, and design thinking techniques to “observe how these processes affected the thoughts, feelings, and experiences of frontline stakeholders.” Most importantly, they engaged with staff members from a variety of disciplines, including physicians, nurses, social workers, and occupational therapists, during the design process—from the beginning.

As Li et al. point out: “…design thinking tools such as empathy mapping helped us more deeply understand how underlying feelings around role clarity and power structures between physician and non-physician members of the care team affected advance care planning workflow. These insights led us to identify key design goals that otherwise would not have surfaced, such as the need to empower non-physician care team members to identify candidate patients and lead the coordination of advance care planning—a task that was enabled by making transparent to the entire care team the list of candidate patients generated by the mortality prediction model each day and creating a workflow for the physician and non-physician team members to discuss these patients with each other about advance care planning needs.”

In addition, Li et al. employed the latest innovations in implementation science and systems engineering. There are well documented principles and techniques that can be brought to bear when AI algorithms are being installed in a hospital or office practice. They enable decision makers to understand the mechanisms involved in bringing about the improvements in advance care planning in the above scenario, for instance. These management tools are also capable of giving decision makers an understanding of the structure, patterns, and processes of workflows that were most useful during the implementation process.

A second AI delivery approach worth consideration is sometimes referred to as vertical deployment. In more traditional industries, products are brought to market only after their initial construction and testing are plugged into a supply chain matrix that includes other critically important, synchronized steps. We recently described this AI vertical deployment in Digital Medicine, a paper co-authored by Joe Zhang, Institute of Global Health Innovation, Imperial College London, and others.

Once again, the emphasis is on cooperation among many diverse players: The goal is to "move beyond academically focused groups to cross-disciplinary teams, where end-users, developers and deployment engineers, and implementation experts, play as significant a role as clinician scientists and data scientists.”

At Mayo Clinic, we have adopted this “one stop shopping” approach using Mayo Clinic Platform’s four key products; Gather, Discover, Validate, and Deliver; which were discussed at length in other publications and on the Mayo Clinic Platform website.