BACKGROUND The National Early Warning Score (NEWS) is widely used to predict patient deterioration, but retrospective validations face intervention bias and emphasise discrimination performance over clinical utility. We aimed to compare the clinical net benefit of NEWS against simplified scoring rules and machine learning models, to determine whether simpler approaches suffice or if complex models offer meaningful advantages, and to evaluate performance heterogeneity across patient subgroups. METHODS In this multicentre retrospective cohort study, we included data from 15 Danish hospitals (ten hospital networks) across the Capital and Zealand regions of Denmark, between Oct 16, 2018, and Oct 16, 2023. We included all adults (aged ≥18 years) with a hospital encounter with at least one NEWS assessment. Demographic and clinical data, NEWS data, and time of death were extracted from hospital and regional electronic health records. Patients were followed up until death or completion of the 24 h observation period after the initial NEWS score was recorded, whichever occurred first. The outcome was all-cause mortality within 24 h of the initial NEWS recording. We compared NEWS against both simpler and more complex approaches for predicting 24 h mortality: simplified NEWS (NEWS without blood pressure and temperature), Demographic Early Warning Score (DEWS; simplified NEWS with age and sex), and a model based on Extreme Gradient Boosting Early Warning Score (XGB-EWS) incorporating vital signs, demographics, laboratory markers, plus medical history embeddings extracted using sentence transformers. We used propensity score weighting to mitigate intervention bias and evaluated performance using net benefit, area under the receiver operating characteristic curve, and calibration. FINDINGS This study included data for 2 080 765 hospital encounters representing 825 200 unique patients over 5 years. The majority (50·9%) of the weighted population was aged 18-65 years, 50·4% were female, 49·6% were male, and 66·2% were admitted to internal and acute medicine clinical specialties. Decision curve analysis in the overall population showed maximum net benefit differences of 1·9 additional correct mortality identifications per 10 000 patients between XGB-EWS and NEWS, and 1·5 per 10 000 patients between NEWS and simplified NEWS. However, stratified analyses revealed substantial heterogeneity: in the high-risk group (initial NEWS ≥7), XGB-EWS provided a maximum gain of 78·1 correct identifications per 10 000 patients compared with NEWS, whereas simplified NEWS resulted in a maximum performance loss of 60·2 per 10 000 patients. Conversely, in the low-risk group (NEWS 0-4), differences were minimal: XGB-EWS provided a maximum gain of 0·9 correct identifications per 10 000 patients compared with NEWS, while simplified NEWS resulted in a maximum loss of 0·1 per 10 000 patients. INTERPRETATION Model complexity produced small average clinical utility gains, concentrated among patients initially classified as high risk by NEWS. These findings support prospective trials of risk-stratified monitoring based on the first NEWS assessment, testing simplified approaches for lower-risk patients while retaining full NEWS or electronic health record models for higher-risk patients. FUNDING Novo Nordisk Foundation.
