SP11.17 The predictive paradigm: examining the validity of a national selection process in surgical training

Abstract Introduction Selection processes should conform to a ‘predictive paradigm’, identifying candidates who will become competent professionals. This study investigates the ability of a centralised, multi-modal national selection process to predict the future performance of surgical trainees. Methods A retrospective analysis of prospectively collected data from successful applicants to the Royal College of Surgeons in Ireland (RCSI) Core Surgical Training (CST) programme from 2016-2020 was conducted. Performance in subsequent assessments, attrition rates, and rates of progression to Higher Specialist Training (HST) were recorded. Selection decisions were based on a total score, composed of technical aptitude assessments, undergraduate academic performance, and a four-station multi-interview. CST performance was assessed using workplace-based and simulation-based assessments. Associations between selection and assessment measures were explored using Pearson correlation, binary logistic regression, and multiple linear regression analyses. Results Data was available for 303 trainees. Composite scores were positively associated with likelihood of progressing to HST (Exp(B) 1.09, 95% CI 1.05 – 1.13, p<0.001), and were significantly associated with performance across all CST assessments (r .23 - .34, P<0.001). On multivariate analysis, technical aptitude scores were the strongest predictor of future operative performance, both in the workplace (B 0.31, 95% CI .14 - .48, p<0.001) and simulated environments (B .57, 95% CI .33 - .81, p<0.001). The ‘interpersonal skills’ interview station was the strongest predictor of performance in simulated non-technical skill assessments (B .55, 95% CI .22 - .87, p<0.001). Conclusion Performance at the time of national selection, measured across technical and non-technical domains in a multi-modal fashion, is associated with future performance in the workplace and in simulation-based assessments.