Validation of a Machine Learning Diagnostic Tool for the Prediction of Sepsis and Critical Illness

Importance: Prompt and accurate diagnosis and risk assessment is a challenge with implications for clinical care of sepsis patients. Objective: To describe the development of the Sepsis ImmunoScore Artificial Intelligence/Machine Learning (AI/ML) algorithm and assess its ability to identify patients with sepsis within 24 hours, and secondary endpoints of critical illness and mortality. Design: Prospective study of adult (age 18 or older) patients from 5 US hospitals enrolled between April 2017 and July 2022. Setting: Multi-center study from 5 hospitals Participants: Inclusion criteria: suspected infection (indicated a blood culture order), emergency department or hospitalized patients, with a corresponding lithium-heparin plasma sample available; exclusion criteria: none. Participants were enrolled into an algorithm development derivation cohort (n=2,366), an internal validation (n=393) cohort, or an external validation cohort (n=698). Main Outcomes and Measures: The primary endpoint was the presence of sepsis (Sepsis-3) within 24 hours of test initiation. Secondary endpoints were clinically relevant metrics of critical illness: length of stay in the hospital, Intensive Care Unit (ICU) admission within 24 hours, use of mechanical ventilation within 24 hours, use of vasopressors within 24 hours, and in-hospital mortality. Results: The overall diagnostic accuracy of the Sepsis ImmunoScore for predicting sepsis was high with an AUC of 0.85 (0.83,0.87) in the derivation cohort, 0.80 (0.74,0.86) in internal validation, and 0.81 (0.77,0.86) in external validation. The Sepsis ImmunoScore was divided into four risk categories with increasing likelihood ratios for sepsis: low 0.1 (0.1,0.2), medium 0.5 (0.3,0.8), high 2.1 (1.8,2.5), very high 8.3 (4.1,17.1). Risk categories also predicted in-hospital mortality rates: low: 0.0% (0.0%, 1.6%), medium: 1.9% (0.4%,5.5%), high: 8.7% (5.7%,12.7%), and very high: 18.2% (7.0%,35.5%) in the external validation cohort. Similar findings were observed for length of stay, ICU utilization, mechanical ventilation and vasopressor use. Conclusions and Relevance: The sepsis ImmunoScore, an AI/ML diagnostic tool, demonstrated high accuracy for predicting sepsis and critical illness that could enable prompt identification of patients at high risk of sepsis and adverse outcomes, which holds promise to inform medical decision making to improve care and outcomes in sepsis. ### Competing Interest Statement Conflict of Interest Disclosures: Zhao, Zhu, Shapiro and Bashir are consultants to Prenosis. Bhargava, Lopez-Espina, Schmalz, Khan, Watson, Uridales, Updike, and Reddy. Jr are employed by Prenosis. Bashir and Shapiro have equity ownership in Prenosis, and Bashir has equity interest in VedaBio. Dr. Shapiro is a consultant for Luminos technologies, Cambridge Medical Technologies, and receives research support from Bluejay diagnostics and Inflammatix. ### Funding Statement Funding/Support: This study was funded in part by the Defense Threat Reduction Agency, National Institutes of Health, Centers for Disease Control and Prevention, National Science Foundation, Biomedical Advanced Research and Development Authority, and Prenosis. Role of the Funder/Sponsor: Prenosis was overall responsible for the design and conduct of the study, collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The other funding agencies had no role. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Ethics committee/IRB of Beth Israel Deaconess Medical Center waived ethical approval for this work Ethics committee/IRB of Jesse Brown VA Medical Center waived ethical approval for this work Ethics committee/IRB of Mercy Health waived ethical approval for this work Ethics committee/IRB of Beaumont - waived ethical approval for this work Ethics committee/IRB of Carle Foundation Hospital waived ethical approval for this work Ethics committee/IRB of OSF Saint Francis Medical Center partially waived ethical approval for this work I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced in the present study are unavailable