Higgins, Thomas; Freeseman-Freeman, Laura; Henson, Kathy; Stark, Maureen Author Information

Ringle, Eric S. MSN, RN; Stark, Maureen M. MSc; Freeseman-Freeman, Laura MA; Henson, Kathy N. BSN, RN Author Information

OBJECTIVES: To develop a model to benchmark mortality in hospitalized patients using accessible electronic medical record data. DESIGN: Univariate analysis and multivariable logistic regression were used to identify variables collected during the first 24 hours following admission to test for risk factors associated with the end point of hospital mortality. Models were built using specific diagnosis (International Classification of Diseases, 9th Edition or International Classification of Diseases, 10th Edition) captured at discharge, rather than admission diagnosis, which may be discordant. Variables were selected based, in part, on prior the Acute Physiology and Chronic Health Evaluation methodology and included primary diagnosis information plus three aggregated indices (physiology, comorbidity, and support). A Physiology Index was created using parsimonious nonlinear modeling of heart rate, mean arterial pressure, temperature, respiratory rate, hematocrit, platelet counts, and serum sodium. A Comorbidity Index incorporates new or ongoing diagnoses captured by the electronic medical record during the preceding year. A Support Index considered 10 interventions such as mechanical ventilation, selected IV drugs, and hemodialysis. Accuracy was determined using area under the receiver operating curve for discrimination, calibration curves, and modified Brier score for calibration. SETTING AND PATIENTS: We used deidentified electronic medical record data from 74,434 adult inpatients (ICU and ward) at 15 hospitals from 2010 to 2013 to develop the mortality model and validated using data for additional 49,752 patients from the same 15 hospitals. A second revalidation was accomplished using data on 83,684 patients receiving care at six hospitals between 2014 and 2016. The model was also validated on a subset of patients with an ICU stay on day 1. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: This model uses physiology, comorbidity, and support indices, primary diagnosis, age, lowest Glasgow Coma Score, and elapsed time since hospital admission to predict hospital mortality. In the initial validation cohort, observed mortality was 4.04% versus predicted mortality 4.12% (Student t test, p = 0.37). In the revalidation using a different set of hospitals, predicted and observed mortality were 2.66% and 2.99%, respectively. Area under the receiver operating curve were 0.902 (0.895–0.909) and 0.884 (0.877–0.891), respectively, and calibration curves show a close relationship of observed and predicted mortalities. In the evaluation of the subset of ICU patients on day1, the area under the receiver operating curve was 0.87, with an observed mortality of 8.78% versus predicted mortality of 8.93% (Student t test, p = 0.52) and a standardized mortality ratio of 0.98 (0.932–1.034). CONCLUSIONS: Variables considered by traditional ICU prognostic models accurately benchmark patient mortality for patients receiving care in multiple hospital locations, not only the ICU. Unlike Acute Physiology and Chronic Health Evaluation, this model relies on electronic medical record data alone and does not require personnel to collect the independent predictor variables. Assessing the model’s utility for benchmarking hospital performance will require prospective testing in a larger representative sample of hospitals.

Higgins, Thomas; Stark, Maureen; Henson, Kathy; Freeseman-Freeman, Laura Author Information

OBJECTIVES:To compare Acute Physiology and Chronic Health Evaluation-IV-adjusted mortality and length of stay outcomes of adult ICU patients who tested positive for coronavirus disease 2019 with patients admitted to ICU with other viral pneumonias including a subgroup with viral pneumonia and concurrent acute respiratory distress syndrome (viral pneumonia-acute respiratory distress syndrome). DESIGN:Retrospective review of Acute Physiology and Chronic Health Evaluation data collected from routine clinical care. SETTING:Forty-three hospitals contributing coronavirus disease 2019 patient data between March 14, and June 17, 2020, and 132 hospitals in the United States contributing data on viral pneumonia patients to the Acute Physiology and Chronic Health Evaluation database between January 1, 2014, and December 31, 2019. PATIENTS AND MEASUREMENTS:One thousand four hundred ninety-one patients with diagnosis of coronavirus disease 2019 infection and 4,200 patients with a primary (n = 2,544) or secondary (n = 1,656) admitting diagnosis of noncoronavirus disease viral pneumonia receiving ICU care. A subset of 202 viral pneumonia patients with concurrent acute respiratory distress syndrome was examined separately. INTERVENTIONS:None. MAIN RESULTS:Mean age was 63.4 for coronavirus disease (p = 0.064) versus 64.1 for viral pneumonia. Acute Physiology and Chronic Health Evaluation-IV scores were similar at 56.7 and 55.0, respectively (p = 0.060), but gender and ethnic distributions differed, as did Pao2 to Fio2 ratio and WBC count at admission. The hospital standardized mortality ratio (95% CI) was 1.52 (1.35-1.68) for coronavirus disease patients and 0.82 (0.75-0.90) for viral pneumonia patients. In the coronavirus disease group, ICU and hospital length of stay were 3.1 and 3.0 days longer than in viral pneumonia patients. Standardized ICU and hospital length of stay ratios were 1.13 and 1.46 in the coronavirus disease group versus 0.95 and 0.94 in viral pneumonia patients. Forty-seven percent of coronavirus disease patients received invasive or noninvasive ventilatory support on their first ICU day versus 65% with viral pneumonia. Ventilator days in survivors were longer in coronavirus disease (10.4 d) than in viral pneumonia (4.3 d) patients, except in the viral pneumonia-acute respiratory distress syndrome subgroup (10.2 d). CONCLUSIONS:Severity-adjusted mortality and length of stay are higher for coronavirus disease 2019 patients than for viral pneumonia patients admitted to ICU. Coronavirus disease patients also have longer time on ventilator and ICU length of stay, comparable with the subset of viral pneumonia patients with concurrent acute respiratory distress syndrome. Mortality and length of stay increase with age and higher scores in both populations, but observed to predicted mortality and length of stay are higher than expected with coronavirus disease patients across all severity of illness levels. These findings have implications for benchmarking ICU outcomes during the coronavirus disease 2019 pandemic.

Higgins, Thomas1; Freeseman-Freeman, Laura2; Henson, Kathy2; Stark, Maureen3 Author Information

Freeseman-Freeman, Laura1; Henson, Kathy1; Stark, Maureen2; Higgins, Thomas3 Author Information

Higgins, Thomas; Freeseman-Freeman, Laura; Henson, Kathy; Stark, Maureen Author Information

Bryant, Corey; Johnson, Alistair; Henson, Kathy; Freeseman-Freeman, Laura; Stark, Maureen; Higgins, Thomas Author Information

BackgroundA recent update of the Mortality Probability Model (MPM)-III found 14% of intensive care patients had age as their only MPM risk factor for hospital mortality. This subgroup had a low mortality rate (2% vs 14% overall), and pronounced differences were noted among elderly patients. This article is an expanded analysis of age-related mortality rates in patients in the ICU.MethodsProject IMPACT data from 135 ICUs for 124,885 patients treated from 2001 to 2004 were analyzed. Patients were stratified as elective surgical, emergency/unscheduled surgical, and medical and then further stratified by age and whether additional MPM risk factors were present or absent.ResultsMortality rose with advancing age within all patient categories. Elective surgical patients without other risk factors were the least likely to die at all ages (0.4% for patients aged 18-29 years to 9.2% for patients aged ≥ 90 years), whereas medical patients with one or more additional risk factors had the highest mortality rate (12.1% for patients aged 18-29 years to 36.0% for patients aged ≥ 90 years). In these two subsets, mortality rates approximately doubled in the elective surgical group among patients aged in their 70s (2.4%), 80s (4.3%), and 90s (9.2%) but rose less dramatically in the medical group (27.0%, 30.7%, and 36.0%, respectively).ConclusionsAlthough mortality increased with age, the risk differed significantly by patient subset, even among elderly patients, which may reflect a selection bias. Advanced age alone does not preclude successful surgical and ICU interventions, although the presence of serious comorbidities decreases the likelihood of survival to discharge for all age groups.

Objectives:To examine the sensitivity of the performance of the latest Mortality Probability Model at intensive care unit admission (MPM0-III) to case-mix variations and to determine how specialized models for these subgroups would affect intensive care unit performance assessment. MPM0-III is an important benchmarking tool for intensive care units in Project IMPACT. Overall, MPM0-III has excellent discrimination and calibration but its performance varies on six common patient subsets. Design:A total of 124,171 patients in six subgroups (complex cardiovascular, trauma, elective surgery, medical, neurosurgery, and emergency surgery) were divided randomly into development (60%) and validation (40%) groups. A logistic regression model was developed to predict hospital mortality for each subgroup, using MPM0-III variables. Model performance was evaluated on the validation sets, using Hosmer-Lemeshow and receiver operating characteristic statistics. Intensive care unit standardized mortality ratios, using the subgroup models and MPM0-III, were compared. A sensitivity analysis was used to identify the occurrence of each subgroup associated with degraded MPM0-III performance. Setting:One hundred thirty-five intensive care units at 98 hospitals participating in Project IMPACT between 2001 and 2004. ICUs with <100 patient records were excluded. Patients:Consecutive intensive care unit patients in the Project IMPACT database who were eligible for MPM0-II scoring. Interventions:None. Measurements and Main Results:Hospital mortality and standardized mortality ratio values by intensive care unit. All six subgroup models had good performance on their validation sets. Intensive care unit standardized mortality ratios calculated with MPM0-III and the subgroup models were nearly identical, with MPM0-III identifying 33 of 135 as significant standardized mortality ratio outliers and the subgroup models identifying 35 of 135, with 33 overlapping. Sensitivity analysis indicated that MPM0-III calibration degraded substantially only when patient mix varied significantly from that of the data set on which MPM0-III was based. Conclusion:We recommend users of MPM make MPM0-III their primary model. Subgroup models may have utility when evaluating highly specialized intensive care units or in research on specific, homogeneous populations.

Objective:To validate performance characteristics of the intensive care unit (ICU) admission mortality probability model, version III (MPM0-III) on Project IMPACT data submitted in 2004 and 2005. This data set was external from the MPM0-III developmental and internal validation data collected between 2001 and 2004. Design:Retrospective analysis of clinical data collected concurrently with care. Setting:One hundred three (103) adult ICUs in North America that voluntarily collect and submit data to Project IMPACT. Subjects:A total of 55,459 patients who were eligible for MPM scoring (age ≥18; first ICU admission for hospitalization, excludes burns, coronary care, and cardiac surgical patients). Interventions:None. Measurements:Prevalence of MPM risk factors and their relationship to hospital mortality; calibration and discrimination of MPM0-III model applied to new data. Main Results:Seventy-eight ICUs contributed data to both this study and the original development set. Fifty-six ICUs from the original MPM0-III study were replaced by 25 new ICUs in this external validation set. Patient characteristics (type of patient, risk factors, and resuscitation status) were similar to the original 2001–2004 cohort, except for slightly more patients on mechanical ventilation at admission (32% vs. 27%, p < 0.01) and the percentage of patients having no MPM0-III risk factors except age (11% vs. 14%, p < 0.01). Observed deaths were 7331 (13.2%) vs. 7456 predicted, yielding a standardized mortality ratio of 0.983, 95% CI (0.963–1.001). Conclusions:MPM0-III calibrates on a new population of 55,459 North American patients who include many patients from new ICUs, which helps confirm that the model is robust and was not overfitted to the development sample. Although Project IMPACT participants change over time, 2004–2005 patient risk factors and their relationship to hospital mortality have not significantly changed. The increase in mechanically ventilated patients and reduction in admissions with no risk factors are trends worth following.

Objective: In 1994, Rapoport et al. published a two-dimensional graphical tool for benchmarking intensive care units (ICUs) using a Mortality Probability Model (MPM0-II) to assess clinical performance and a Weighted Hospital Days scale (WHD-94) to assess resource utilization. MPM0-II and WHD-94 do not calibrate on contemporary data, giving users of the graph an inflated assessment of their ICU’s performance. MPM0-II was recently updated (MPM0-III) but not the model for predicting resource utilization. The objective was to develop a new WHD model and revised Rapoport-Teres graph. Design: Multicenter cohort study. Setting: One hundred thirty-five ICUs in 98 hospitals participating in Project IMPACT. Patients: Patients were 124,855 MPM0-II eligible Project IMPACT patients treated between March 2001 and June 2004. Interventions: None. Measurements and Main Results: WHD was redefined as 4 units for the first day of each ICU stay, 2.5 units for each additional ICU day, and 1 unit for each non-ICU day after the first ICU discharge. Stepwise linear regression was used to construct a model to predict ICU-specific log average WHD from 39 candidate variables available in Project IMPACT. The updated WHD model has four independent variables: percent of patients dying in the hospital, percent of unscheduled surgical patients, percent of patients on mechanical ventilation within 1 hr of ICU admission, and percent discharged from the ICU to an external post-acute care facility. The first three variables increase average WHD and the last decreases it. The new model has good performance (R2 = 0.47) and, when combined with MPM0-II, provides a well-calibrated Rapoport-Teres graph. Conclusions: A new WHD model has been derived from a large, contemporary critical care database and, when used with MPM0-III, updates a popular method for benchmarking ICUs. Project IMPACT participants will likely perceive a decline in their ICU performance coordinates due to the recalibrated graph and should instead focus on their unit’s performance relative to their peers.

OBJECTIVE:To update the Mortality Probability Model at intensive care unit (ICU) admission (MPM0-II) using contemporary data. DESIGN:Retrospective analysis of data from 124,855 patients admitted to 135 ICUs at 98 hospitals participating in Project IMPACT between 2001 and 2004. Independent variables considered were 15 MPM0-II variables, time before ICU admission, and code status. Univariate analysis and multivariate logistic regression were used to identify risk factors associated with hospital mortality. SETTING:One hundred thirty-five ICUs at 98 hospitals. PATIENTS:Patients in the Project IMPACT database eligible for MPM0-II scoring. INTERVENTIONS:None. MEASUREMENTS AND MAIN RESULTS:Hospital mortality rate in the current data set was 13.8% vs. 20.8% in the MPM0-II cohort. All MPM0-II variables remained associated with mortality. Clinical conditions with high relative risks in MPM0-II also had high relative risks in MPM0-III. Gastrointestinal bleeding is now associated with lower mortality risk. Two factors have been added to MPM0-III: "full code" resuscitation status at ICU admission, and "zero factor" (absence of all MPM0-II risk factors except age). Seven two-way interactions between MPM0-II variables and age were included and reflect the declining marginal contribution of acute and chronic medical conditions to mortality risk with increasing age. Lead time before ICU admission and pre-ICU location influenced individual outcomes but did not improve model discrimination or calibration. MPM0-III calibrates well by graphic comparison of actual vs. expected mortality, overall standardized mortality ratio (1.018; 95% confidence interval, 0.996-1.040) and a low Hosmer-Lemeshow goodness-of-fit statistic (11.62; p = .31). The area under the receiver operating characteristic curve was 0.823. CONCLUSIONS:MPM0-II risk factors remain relevant in predicting ICU outcome, but the 1993 model significantly overpredicts mortality in contemporary practice. With the advantage of a much larger sample size and the addition of new variables and interaction effects, MPM0-III provides more accurate comparisons of actual vs. expected ICU outcomes.

PURPOSE: The Mortality Probability Model on ICU admission (MPM0) is used by Project IMPACT as a benchmarking tool. A recent update (MPM0-III) has been validated and shown to perform well on a large dataset. New MPM models based on specific patient subsets have been constructed to improve assessment with specialized ICU populations.

Nathanson, Brian; Higgins, Thomas L; Teres, Daniel; Copes, Wayne S; Kramer, Andrew; Stark, Maureen Author Information

It is known that the forms of physical education practised among ancient Greek communities reflected the differential importance attached to military and artistic activities.1 Similarly, today, variations in the relative pre-eminence of collective over individual sports are likely to reflect contrasts in the socio-economic and cultural profiles of contemporary societies.2 Moreover, modes of social participation in a particular game should be affected by the patterns of interaction prevailing in the society at large.