Using Automated Continuous Instrument Tracking to Benchmark Simulated Laparoscopic Performance and Personalize Training

OBJECTIVE: Laparoscopic simulation is widely used in surgical training. However, the impact of training on performance is difficult to assess. Observation is time-intensive and subjective. SurgTrac laparoscopic box-trainer instrument tracking software provides continuous, automated, real-time, objective performance feedback. We used this data to assess the relationship between task attempts and performance. We assessed whether improvement in performance with repetition could be modeled in learning curves that might be used for benchmarking. DESIGN: Anonymized SurgTrac data for performances undertaken between 10/2016 and 05/2019 were retrospectively extracted. The thread transfer task, a basic instrument handling task, was assessed. Task duration and instrument-based metrics were analyzed; total distance travelled by instrument tips, average speed, average acceleration, and the ratio of movements between the left and right hands. Curve estimation regression was used to assess the relationship between attempt number and metrics for pooled data across the entire cohort of users and amongst individual users with >= 50 attempts. Threshold for significance p = 0.05. SETTING: SurgTrac has generated the largest available database of performances in box trainer simulated tasks with 64,000 activities performed by over 1450 users in 77 countries to date. PARTICIPANTS: Data was derived from the unselected world-wide cohort of SurgTrac users. No participants were excluded. RESULTS: Five hundred seventy-eight users performed 13,027 attempts in the thread transfer task. Across the entire cohort, SurgTrac performance metrics were significantly associated with attempt number. Task duration and total distance decreased with attempt number. This benefit persisted across 100 attempts. Ambidexterity increased with attempt number. Individual candidate performance improved in line with predicted learning curves for better performing candidates. CONCLUSIONS: We analyzed the largest database of simulated laparoscopic task performances. Performance improves with practice. Using learning curves derived from peer-group performances as benchmarks, users may be regularly and objectively assessed to support personalization of training. (C) 2020 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.