Assessment of surgical competence for neck dissection: a pilot study

Background: Progressive implementation of the milestone competence-based curriculum has created a need for new objective and validated means to assess resident surgical proficiency. A previous systematic review of the literature by our group has highlighted a shortage of tools assessing surgical competence in oncologic procedures in otolaryngology - head and neck surgery. Methods: We developed a procedure-specific assessment tool for neck dissection using a modified Delphi method. The 2-part design was modelled on the previously validated Objective Structured Assessment of Technical Skills checklist. The tool was then validated through a 1-year multicentric prospective study in collaboration with the residents and faculty from our academic centre. Additionally, we developed an online survey to assess the acceptability by residents and staff before and after the validation studies. Results: A total of 29 evaluations were completed throughout the 2016-2017 academic year. Acceptability ranked high for both residents and staff, with a single discrepancy in responses regarding a potential formative as opposed to summative use of the tool. Validation study results showed significantly higher checklist scores among senior residents than junior residents, as well as a significant score progression over time (p < 0.05). Trends in scores on the task-specific tool correlated highly to results obtained on a validated global rating scale (p < 0.05). Conclusion: The first tool assessing surgical competence in oncologic otolaryngology - head and neck surgery has been developed and shows promising validity.