External validation and insights about the calibration of the return of spontaneous circulation after cardiac arrest (RACA) score

Background The return of spontaneous circulation (ROSC) after cardiac arrest (RACA) score was developed as a tool to predict ROSC probability (pROSC) based on easily available information and it could be useful to compare the performances of different EMS agencies or the effects of eventual interventions.We performed an external validation of the RACA score in a cohort of out of hospital cardiac arrest (OHCA) patients managed by the EMS of the metropolitan city of Bologna, Italy. Methods We analyzed data from 2,310 OHCA events prospectively collected between January 2009 and June 2021. Discrimination was assessed with the area under the ROC curve (AUROC), while the calibration belts were used for the comparison of observed versus expected ROSC rates. The AUROCs from our cohort and other validation cohorts were compared using a studentized range test. Results The AUROC for the study population was 0.691, comparable to that described by previous validation studies. Despite an acceptable overall calibration, we found a poor calibration for asystole and low pROSC ranges in PEA and shockable rhythms. The model showed a good calibration for patients aged over 80, while no differences in performance were found when evaluating events before and after the implementation of 2015 ERC guidelines. Conclusions Despite AUROC values being similar in different validation studies for RACA score, we suggest separating the different rhythms when assessing ROSC probability with the RACA score, especially for asystole.