Validation of an Objective Assessment Tool for Tonsillectomy in Otolaryngology Resident Training

ObjectiveCreate and validate an objective structured assessment of technical skills (OSATS) for otolaryngology residents learning how to perform a tonsillectomy.Study DesignMulticenter prospective longitudinal validation study.MethodsA multi‐institutional study at six tertiary academic otolaryngology residency programs from July 2009 to May 2012. Using the modified Delphi technique, a panel of pediatric otolaryngologists created a tonsillectomy task‐based checklist (TBC) for a tonsil OSATS using a 5‐point Likert‐type scale. Residents were assessed by pediatric otolaryngology staff at the time of surgery with the TBC and a global rating scale. Procedure time, patient age, number of previously performed tonsillectomies, and surgical technique were also collected.ResultsOne hundred sixty‐seven tonsil OSATS were completed for 38 residents, and competency was recorded for 99 (59.2%). Residents scored as competent had performed significantly more previous tonsillectomies than those deemed noncompetent, 44.4 ± 35.6 and 13.5 ± 11.6, respectively (P < .001). The mean overall score on the tonsil TBC was 4.0 ± 0.8 and 2.6 ± 1.0 for competent and noncompetent, respectively (P < .001). Higher number of tonsillectomies performed and mean tonsil TBC score significantly increased the likelihood of competency (P < .001). Each additional tonsillectomy performed increased the likelihood of achieving competency by 6.3% (P = .006, 95% confidence interval (CI): 1.330–1.110), and each 1.0 point increase in mean tonsil TBC score increased the likelihood of competency by a factor of 2.71 (P = .006, 95% CI:1.330–5.513). There is a 95% likelihood of competency at 48 tonsillectomies or a tonsil TBC score of 4.91.ConclusionThe tonsil OSATS is a valid and feasible instrument to assess resident competency with tonsillectomy and provides timely objective feedback.Level of Evidence4. Laryngoscope, 131:E359–E366, 2021