Early Recognition of Sepsis Looks Much More Doable Now
Two recent studies strongly suggest that sepsis predictive algorithms may in fact be ready for “prime time.”
By John Halamka, M.D., president, Mayo Clinic Platform, and Paul Cerrato, senior research analyst and communications specialist, Mayo Clinic Platform
Among the many challenges that clinicians face when caring for critically ill patients is determining how to recognize the onset of sepsis before it’s too late. More than 1.7 million Americans develop this life-threatening complication annually, at least 350,000 adults who develop it die during hospitalization or are discharged to hospice, and 1 in 3 patients who die in a hospital had sepsis during their stay. Given these troubling statistics, it’s essential that the condition be detected early on, while it is often more manageable. Studies suggest, for instance, that early treatment with broad spectrum IV antibiotics reduces its morbidity and mortality. Unfortunately, because sepsis can present so differently in individual patients, it is often overlooked until it has progressed too far.
Data scientists have been attempting to address this problem in recent years, with mixed results. Epic Systems’ sepsis algorithm was said to have a predictive rating of 0.76 to 0.83. (The rating is derived from a metric called area under the curve (AUC); a 1.0 rating means that the algorithm is 100 per cent accurate in detecting the complication early on.) However, a STAT analysis revealed that over time the predictive value of the algorithm had dropped significantly, the result of unanticipated changes in patient characteristics over time, which had been caused by changes in the demographics of patients during the COVID-19 pandemic. Other investigators have reported more positive findings that suggest AI-based algorithms can improve sepsis diagnosis and management, but most have been retrospective in nature—until recently.
Roy Adams and his colleagues from Johns Hopkins University conducted a prospective evaluation of a sepsis prediction model called the Targeted Real-Time Early Warning System (TREWS) in five hospitals over a two-year period. Adams et al. state: “Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within three hours of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within three hours. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk.” More specifically, among more than 6,800 patients with sepsis, mortality was 19% lower when clinicians received confirmation of an alert within three hours, when compared to clinicians who did not receive such timely notification.
Katharine Henry, also with Johns Hopkins University, and her colleagues, used a combination of retrospective and prospective analysis to evaluate the TREWS model, and likewise reported positive results. 89% of alerts that physicians received were evaluated and 38% were confirmed. Each of the five hospitals enrolled in the program reported a sensitivity of about 80% using site-specific historical patient data. On average, using the model enabled clinicians to administer antibiotics 1.85 hours longer for alerts that were evaluated within three hours, when compared to alerts that took earlier to evaluate.
While these results paint a vivid picture of what critical disease management will look like in the near future, such AI-enabled predictive algorithms are not without their own set of problems. David Bates and Ania Syrowatka from Harvard Medical School point out: “To realize large-scale benefits from technologies such as the TREWS, these tools will have to be implemented across very diverse health systems. However, hospitals are already struggling to implement and maximize benefits from such technologies. Many promising algorithms have been developed by different companies to address specific issues, but it is very challenging for hospitals to work with large numbers of vendors.”
Addressing these challenges is a top priority for the Coalition for Health AI, the mission of which is to provide guidelines regarding an ever-evolving landscape of health AI tools to ensure high quality care, increase credibility amongst users, and meet health care needs.