Introduction: Previous studies deriving and validating triage scores for patients with suspected COVID-19 in Emergency Department settings have been conducted in high- or middle -income settings. We assessed eight triage scores' accuracy for death or organ support in patients with suspected COVID-19 in Sudan. Methods: We conducted an observational cohort study using Covid-19 registry data from eight emergency unit isolation centres in Khartoum State, Sudan. We assessed performance of eight triage scores including: PRIEST, LMIC-PRIEST, NEWS2, TEWS, the WHO algorithm, CRB-65, Quick COVID-19 Severity Index and PMEWS in suspected COVID-19. A composite primary outcome included death, ventilation or ICU admission. Results: In total 874 (33.84 %, 95 % CI:32.04 % to 35.69 %) of 2,583 patients died, required intubation/noninvasive ventilation or HDU/ICU admission . All risk -stratification scores assessed had worse estimated discrimination in this setting, compared to studies conducted in higher -income settings: C -statistic range for primary outcome: 0.56-0.64. At previously recommended thresholds NEWS2, PRIEST and LMIC-PRIEST had high estimated sensitivities (>= 0.95) for the primary outcome. However, the high baseline risk meant that lowrisk patients identified at these thresholds still had a between 8 % and 17 % risk of death, ventilation or ICU admission. Conclusion: None of the triage scores assessed demonstrated sufficient accuracy to be used clinically. This is likely due to differences in the health care system and population (23 % of patients died) compared to higher -income settings in which the scores were developed. Risk -stratification scores developed in this setting are needed to provide the necessary accuracy to aid triage of patients with suspected COVID-19.
BackgroundUneven vaccination and less resilient health care systems mean hospitals in LMICs are at risk of being overwhelmed during periods of increased COVID-19 infection. Risk-scores proposed for rapid triage of need for admission from the emergency department (ED) have been developed in higher-income settings during initial waves of the pandemic.MethodsRoutinely collected data for public hospitals in the Western Cape, South Africa from the 27th August 2020 to 11th March 2022 were used to derive a cohort of 446,084 ED patients with suspected COVID-19. The primary outcome was death or ICU admission at 30 days. The cohort was divided into derivation and Omicron variant validation sets. We developed the LMIC-PRIEST score based on the coefficients from multivariable analysis in the derivation cohort and existing triage practices. We externally validated accuracy in the Omicron period and a UK cohort.ResultsWe analysed 305,564 derivation, 140,520 Omicron and 12,610 UK validation cases. Over 100 events per predictor parameter were modelled. Multivariable analyses identified eight predictor variables retained across models. We used these findings and clinical judgement to develop a score based on South African Triage Early Warning Scores and also included age, sex, oxygen saturation, inspired oxygen, diabetes and heart disease. The LMIC-PRIEST score achieved C-statistics: 0.82 (95% CI: 0.82 to 0.83) development cohort; 0.79 (95% CI: 0.78 to 0.80) Omicron cohort; and 0.79 (95% CI: 0.79 to 0.80) UK cohort. Differences in prevalence of outcomes led to imperfect calibration in external validation. However, use of the score at thresholds of three or less would allow identification of very low-risk patients (NPV ≥0.99) who could be rapidly discharged using information collected at initial assessment.ConclusionThe LMIC-PRIEST score shows good discrimination and high sensitivity at lower thresholds and can be used to rapidly identify low-risk patients in LMIC ED settings.
COVID-19 infection rates remain high in South Africa. Clinical prediction models may be helpful for rapid triage, and supporting clinical decision making, for patients with suspected COVID-19 infection. The Western Cape, South Africa, has integrated electronic health care data facilitating large-scale linked routine datasets. The aim of this study was to develop a machine learning model to predict adverse outcome in patients presenting with suspected COVID-19 suitable for use in a middle-income setting. A retrospective cohort study was conducted using linked, routine data, from patients presenting with suspected COVID-19 infection to public-sector emergency departments (EDs) in the Western Cape, South Africa between 27th August 2020 and 31 st October 2021. The primary outcome was death or critical care admission at 30 days. An XGBoost machine learning model was trained and internally tested using split-sample validation. External validation was performed in 3 test cohorts: Western Cape patients presenting during the Omicron COVID-19 wave, a UK cohort during the ancestral COVID-19 wave, and a Sudanese cohort during ancestral and Eta waves. A total of 282,051 cases were included in a complete case training dataset. The prevalence of 30-day adverse outcome was 4.0%. The most important features for predicting adverse outcome were the requirement for supplemental oxygen, peripheral oxygen saturations, level of consciousness and age. Internal validation using split-sample test data revealed excellent discrimination (C-statistic 0.91, 95% CI 0.90 to 0.91) and calibration (CITL of 1.05). The model achieved C-statistics of 0.84 (95% CI 0.84 to 0.85), 0.72 (95% CI 0.71 to 0.73), and 0.62, (95% CI 0.59 to 0.65) in the Omicron, UK, and Sudanese test cohorts. Results were materially unchanged in sensitivity analyses examining missing data. An XGBoost machine learning model achieved good discrimination and calibration in prediction of adverse outcome in patients presenting with suspected COVID19 to Western Cape EDs. Performance was reduced in temporal and geographical external validation.
BackgroundTools proposed to triage ED acuity in suspected COVID-19 were derived and validated in higher income settings during early waves of the pandemic. We estimated the accuracy of seven risk-stratification tools recommended to predict severe illness in the Western Cape, South Africa. MethodsAn observational cohort study using routinely collected data from EDs across the Western Cape, from 27 August 2020 to 11 March 2022, was conducted to assess the performance of the PRIEST (Pandemic Respiratory Infection Emergency System Triage) tool, NEWS2 (National Early Warning Score, version 2), TEWS (Triage Early Warning Score), the WHO algorithm, CRB-65, Quick COVID-19 Severity Index and PMEWS (Pandemic Medical Early Warning Score) in suspected COVID-19. The primary outcome was intubation or non-invasive ventilation, death or intensive care unit admission at 30 days. ResultsOf the 446 084 patients, 15 397 (3.45%, 95% CI 34% to 35.1%) experienced the primary outcome. Clinical decision-making for inpatient admission achieved a sensitivity of 0.77 (95% CI 0.76 to 0.78), specificity of 0.88 (95% CI 0.87 to 0.88) and the negative predictive value (NPV) of 0.99 (95% CI 0.99 to 0.99). NEWS2, PMEWS and PRIEST scores achieved good estimated discrimination (C-statistic 0.79 to 0.82) and identified patients at risk of adverse outcomes at recommended cut-offs with moderate sensitivity (>0.8) and specificity ranging from 0.41 to 0.64. Use of the tools at recommended thresholds would have more than doubled admissions, with only a 0.01% reduction in false negative triage. ConclusionNo risk score outperformed existing clinical decision-making in determining the need for inpatient admission based on prediction of the primary outcome in this setting. Use of the PRIEST score at a threshold of one point higher than the previously recommended best approximated existing clinical accuracy.
Aims, Objectives and BackgroundUneven vaccination in low- and middle-income settings and less resilient health care provision mean that emergency health care systems may still be at risk of being overwhelmed during periods of increased COVID-19 infection. Risk stratification tools proposed to allow rapid triage of need for admission in ED settings have almost exclusively been developed and validated in high-income settings during early waves of the pandemic.Our study aimed to estimate the accuracy of risk-stratification tools recommended to predict severe illness in adults with suspected COVID-19 infection in the Western Cape of South Africa.Method and DesignAn observational cohort study using routinely electronically collected clinical information in all state-run hospitals in the Western Cape between 27th August 2020 and 11th March 2022 was conducted to assess performance of the PRIEST tool, NEWS2, the WHO algorithm, CRB-65, TEWS, Quick Covid Severity Index and PMEWS in patients with suspected COVID-19. The primary outcome was death, respiratory support or ICU admission.Abstract 1482 Figure 1Performance of tools predicting composite primary outcome for total study periodAbstract 1482 Figure 2Performance of tools predicting composite primary outcome for the Omicron periodAbstract 1482 Table 1Triage tool diagnostic accuracy statistics (95% CI) for predicting any adverse outcome (entire study period)ToolN*C-statisticThresholdN (%) above thresholdSensitivitySpecificityPPVNPVCRB-65432,5840.70(0.70, 0.71)>0102,964 (23.8%)0.61(0.61, 0.61)0.78(0.77, 0.78)0.09(0.09, 0.09)0.98(0.98, 0.98)NEWS2433,1010.80(0.79, 0.80)>1178835 (41.3%)0.83(0.83, 0.83)0.6(0.6,0.6)0.07(0.07–0.07)0.99(0.99, 0.99)PMEWS438,8100.79(0.79, 0.79)>2199,386 (45.4%)0.85(0.85, 0.85)0.56(0.56, 0.56)0.06(0.06, 0.07)0.99 (0.99,0.99)PRIEST438,8800.82(0.82, 0.82)>4158,893 (36.2%)0.83(0.83, 0.83)0.65 (0.65,0.66)0.08(0.08, 0.08)0.99(0.99, 0.99)WHO437,8500.71(0.71, 0.72)>0235,775 (53.8%)0.82(0.81, 0.82)0.47(0.47, 0.47)0.05(0.05, 0.05)0.99(0.99, 0.99)TEWS432,6120.72(0.71, 0.72)>2134,097 (31%)0.62(0.62, 0.62)0.70(0.70, 0.70)0.07(0.07, 0.07)0.98(0.98, 0.98)Quick COVID446,0880.70(0.69, 0.70)>335,145 (7.9%)0.33(0.33, 0.33)0.93(0.93, 0.93)0.14(0.14, 0.14)0.98(0.98, 0.98)*Patients with <3 parameters were excluded from analysis when estimating performanceAbstract 1482 Table 2Triage tool diagnostic accuracy statistics (95% CI) for predicting any adverse outcome (Omicron period)ToolN*C-statisticThresholdN (%) above thresholdSensitivitySpecificityPPVNPVCRB-65136,9610.69(0.68, 0.70)>031,373 (22.9%)0.59(0.59, 0.59)0.78(0.78, 0.78)0.05(0.05, 0.05)0.99(0.99, 0.99)NEWS2137,1250.77(0.76, 0.78)>176,183 (55.6%)0.87(0.87, 0.87)0.45(0.45, 0.45)0.03(0.03, 0.03)0.99(0.99, 0.99)PMEWS138,9540.76(0.75, 0.76)>259,876 (43.1%)0.80(0.80, 0.80)0.58(0.58, 0.58)0.04(0.04, 0.04)0.99(0.99, 0.99)PRIEST158,8930.78(0.77, 0.79)>446,529 (33.5%)0.75(0.75, 0.75)0.67(0.67, 0.67)0.04(0.04, 0.04)0.99(0.99, 0.99)WHO138,6660.62(0.61, 0.63)>072,599 (52.4%)0.70(0.70, 0.70)0.48(0.48, 0.48)0.03(0.03, 0.03)0.99(0.99, 0.99)TEWS136,9670.73(0.72, 0.74)>239,509 (28.8%)0.64(0.64, 0.64)0.72(0.72, 0.72)0.04(0.04, 0.04)0.99(0.99, 0.99)Quick COVID1405200.61(0.60, 0.63)>38,210(6.4%)0.17(0.17, 0.17)0.94(0.94, 0.94)0.06(0.06, 0.06)0.98(0.98, 0.98)*Patients with <3 parameters were excluded from analysis when estimating performanceResults and ConclusionOf the 446,084 patients, 15,397 patients (3.45%, 95% CI:34% to 35.1%) experienced the primary outcome. Figure 1 presents the ROC curves for the triage tools for the total study period and figure 2 for the period of the Omicron wave. NEWS2, PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcomes at recommended cut-offs with moderate sensitivity (>0.8) and specificity ranging from 0.47 (NEWS2) to 0.65 (PRIEST tool). The low prevalence of the primary outcome, especially in the Omicron period, meant use of these tools would have more than doubled admissions with only a small reduction in risk of false negative triage.Triage tools developed specifically in low- and middle-income settings may be needed to provide accurate risk prediction. Existing triage tools may need to be used at varying thresholds to reflect different baseline-line risks of adverse outcomes in different settings.
Background: Inappropriate dispatch of urgent ambulances by call-centre personnel causes an unnecessary drain on existing resources. How often these urgent dispatches are appropriate has not been evaluated in the lower-middle-income setting, nor have factors been assessed that contribute to these decisions. This study aimed to establish the rates of pre-hospital over-triage in Cape Town, South Africa, and to assess the call-centre decision-making processes. Methods: This was a descriptive, retrospective study examining all urgent ambulance dispatches made from a large public sector ambulance call centre in Cape Town over a single month. This urgent dispatch was then compared to the on-scene South African Triage Scale (SATS) score assigned by the pre-hospital personnel to assess which cases were 'over-triaged' by the call taker. Factors potentially contributing to the call taker's decision were also analysed and included the time of day, nature of presenting complaint, and the call taker's training and experience – all of which may have affected the rates of over-triage.Results: In the course of one month in 2017, 4,169 urgent calls were assessed; of these, 2,701 were over-triaged (58.48%). The over-triage rate was similar during the day (58.02%) and night (59.11%). The most regularly over-triaged complaint was obstetric and gynaecological (84.87%), followed by motor vehicle accidents (65.70%); the lowest rate was for cardiac call-outs (47.12%). We reviewed the 38 highest workload call takers, and found no statistically significant factors that contributed to higher levels of over-triage when reviewing higher levels of training (Ambulance Emergency Assistant 62.16%, no medical training 59.42%; p=0.669), more years as a call taker (< 2 years 59.32%, > 5 years 60.23%; p=0.932), and more years working in the field (0 years 59.36%, > 5 years 63.66%; p=0.305).Conclusion: The rates of pre-hospital over-triage in Cape Town are marginally lower than those described internationally. The nature of the complaint had a strong impact on these rates, notably in terms of trauma and gynaecological disorders. The call taker's training and years of experience did not have a statistically significant impact on decision-making.