Healthcare AI Has to Earn Clinicians’ Trust

The Coalition for Health AI was created to help end users separate trustworthy, useful technology from technology that only seems to move us in the right direction.

By John Halamka, M.D., President, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform.

“Pink Pills for Pale People.” You probably don’t recall the ad for this patent medicine, which was very popular in the early 20^th century as a cure for nervous headaches, rheumatism, ataxia, palpitations, and pale complexion. Like many patent medicines being marketed at the time, the ad was based on a kernel of truth. It contained iron sulfate, which would have relieved the pallor brought on by iron deficiency anemia. Unfortunately, like many other “miracle drugs” of that era, the scientific foundation upon which the Pink Pills rested wasn’t very trustworthy. The same dilemma now faces many AI-based algorithms coming to market.

As we have discussed in other publications, many of these digital tools don’t solve the problems they were designed to solve or do more harm than good. That’s one of the reasons the Coalition for Health AI (CHAI) was developed. CHAI is a community of academic health systems, organizations, and expert artificial intelligence (AI) and data science practitioners. These members have come together to harmonize standards and reporting for health AI and educate end-users on how to evaluate these technologies to drive their adoption.

 CHAI recently published its Blueprint for Trustworthy AI Implementation Guidance and Assurance for Healthcare to build confidence in a set of principles that can be relied on by developers, healthcare providers, and other stakeholders. But like all organizations involved in digital health, we are aware of the need to earn the trust of all these stakeholders. One way we accomplish that objective is to involve a wide variety of thought leaders from several segments of the healthcare ecosystem. That includes data scientists, software engineers, vendors, physicians, nurses, clinicians in training, insurers, patient advisory groups, governmental regulators, research funders, educators, and many others. 

Similarly, the Blueprint points out: “This work has brought together a collaboration across a number of institutions with expertise in different areas relevant to this effort to attain sufficiently broad coverage. The goal was to ensure applicability to a wide range of clinical AI-based systems and thus facilitate more widespread adoption. Some of the institutions that have already published guidelines (i.e., Duke, Stanford, Johns Hopkins) are part of this work. While there are current efforts to develop core ingredients for AI/ML for specific medical applications like cardiac software and medical devices, the clinical AI/ML community would benefit from an approach that could be applied to AI-based clinical algorithms for various uses (e.g., diagnostic, prognostic) and clinical subdomains (e.g., oncology, cardiology).”

In an earlier blog, we discussed the role CHAI can play in validating AI algorithms, one of the primary reasons the organization was developed. But validation is only one part of the formula for reliable, safe algorithms. We also realize that healthcare AI is a moving target, with new technologies rapidly emerging month by month. Thus, the need for sandboxes to test these emerging innovations, as well as assurance accreditation labs and technical assistance services.   

The need for reliable, safe digital tools that can improve the diagnostic process and foster new treatment options has never been greater.  The need for such tools has been recognized by technology companies and government officials alike. In a recent news conference, for instance, Senate Majority Leader Chuck Schumer announced a plan to regulate AI across the board, explaining that the plan “will begin with at least nine panels to identify and discuss the hardest questions that regulations on AI will have to answer, including how to protect workers, national security and copyright, and to defend against “doomsday scenarios.” The panels will be composed of experts from industry, academia, and civil society, with the first sessions taking place in September….”

That plan is consistent with the goals and objectives of CHAI, which hopes to energize a coalition of the willing to join us in our quest to earn the trust of all healthcare stakeholders.