Accuracy of automated computer aided-risk scoring systems to estimate the risk of COVID-19 and in-hospital mortality: a retrospective cohort study

Objectives Although a set of computer-aided risk scoring systems (CARSS), that use the National Early Warning Score and routine blood tests results, have been validated for predicting in-hospital mortality and sepsis in unplanned admission to hospital, little is known about their performance for COVID-19 patients. We compare the performance of CARSS in unplanned admissions with COVID-19 during the first phase of the pandemic. Design a retrospective cross-sectional study Setting Two acute hospitals (Scarborough and York) are combined into a single dataset and analysed collectively. Participants Adult (>=18 years) non-elective admissions discharged between 11-March-2020 to 13-June-2020 with an index NEWS electronically recorded within ±24 hours. We assessed the performance of all four risk score (for sepsis: CARS\_N, CARS\_NB; for mortality: CARM\_N, CARM\_NB) according to discrimination (c-statistic) and calibration (graphically) in predicting the risk of COVID-19 and in-hospital mortality. Results The risk of in-hospital mortality following emergency medical admission was 8.4% (500/6444) and 9.6% (620/6444) had a diagnosis of COVID-19. For predicting COVID-19 admissions, the CARS\_N model had the highest discrimination 0.73 (0.71 to 0.75) and calibration slope 0.81 (0.72 to 0.89). For predicting in-hospital mortality, the CARM\_NB model had the highest discrimination 0.84 (0.82 to 0.75) and calibration slope 0.89 (0.81 to 0.98). Conclusions Two of the computer-aided risk scores (CARS\_N and CARM\_NB) are reasonably accurate for predicting the risk of COVID-19 and in-hospital mortality, respectively. They may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions because they are automated and require no additional data collection. Article Summary ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This research was supported by the Health Foundation. The Health Foundation is an independent charity working to improve the quality of health care in the UK. This research was supported by the National Institute for Health Research (NIHR) Yorkshire and Humberside Patient Safety Translational Research Centre (NIHR YHPSTRC). The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: This study used de-identified data and received ethical approval from the Health Research Authority (HRA) and Health and Care Research Wales (HCRW) (reference number 19/HRA/0548). All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes Our data sharing agreement with NHS York hospital trust does not permit us to share this data with other parties. Nonetheless, if anyone is interested in the data, then they should contact the R&D offices in the first instance.