Neurosurgical Skills Assessment: Measuring Technical Proficiency in Neurosurgery Residents through Intraoperative Video Evaluations

OBJECTIVES: Although technical skills are fundamental in neurosurgery, there is little agreement on how to describe, measure, or compare skills among surgeons. The primary goal of this study was to develop a quantitative grading scale for technical surgical performance that distinguishes operator skill when graded by domain experts (residents, attendings, and nonsurgeons). Scores provided by raters should be highly reliable with respect to scores from other observers. METHODS: Neurosurgery residents were fitted with a head-mounted video camera while performing craniotomies under attending supervision. Seven videos, 1 from each postgraduate year (PGY) level (1-7), were anonymized and scored by 16 attendings, 8 residents, and 7 nonsurgeons using a grading scale. Seven skills were graded: incision, efficiency of instrument use, cauterization, tissue handling, drilling/craniotomy, confidence, and training level. RESULTS: A strong correlation was found between skills score and PGY year (P < 0.001, analysis of variance). Junior residents (PGY 1-3) had significantly lower scores than did senior residents (PGY 4-7, P < 0.001, t test). Significant variation among junior residents was observed, and senior residents' scores were not significantly different from one another. Interrater reliability, measured against other observers, was high (r=0.581 +/- 0.245, Spearman), as was assessment of resident training level (r =0.583 +/- 0.278, Spearman). Both variables were strongly correlated (r=0.90, Pearson). Attendings, residents, and nonsurgeons did not score differently (P=0.46, analysis of variance). CONCLUSIONS: Technical skills of neurosurgery residents recorded during craniotomy can be measured with high interrater reliability. Surgeons and nonsurgeons alike readily distinguish different skill levels. This type of assessment could be used to coach residents, to track performance over time, and potentially to compare skill levels. Developing an objective tool to evaluate surgical performance would be useful in several areas of neurosurgery education.