Importance of cognitive load for automated error detection in robot-assisted surgery education

You have accessJournal of UrologyCME1 Apr 2023PD30-03 IMPORTANCE OF COGNITIVE LOAD FOR AUTOMATED ERROR DETECTION IN ROBOT-ASSISTED SURGERY EDUCATION Rafal Kocielnik, Jasper A. Laca, Elyssa Y. Wong, Timothy N. Chu, Sidney I. Roberts, Sandra Marshall, Anima Anandkumar, and Andrew J. Hung Rafal KocielnikRafal Kocielnik More articles by this author , Jasper A. LacaJasper A. Laca More articles by this author , Elyssa Y. WongElyssa Y. Wong More articles by this author , Timothy N. ChuTimothy N. Chu More articles by this author , Sidney I. RobertsSidney I. Roberts More articles by this author , Sandra MarshallSandra Marshall More articles by this author , Anima AnandkumarAnima Anandkumar More articles by this author , and Andrew J. HungAndrew J. Hung More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003316.03AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Cognitive Load (CL) can affect surgeon performance and the ability of training surgeons to receive feedback. We investigate the value of CL for detecting surgical errors in robot-assisted surgery (RAS) with a machine learning (ML)-based model. METHODS: We use 2 datasets (DS1, DS2) where 24 and 44 surgeons, respectively, performed a simulated dry-lab tissue dissection task on a live daVinci Xi surgical robot. The data contains video and per-second Index of Cognitive Activity (ICA), derived from pupillary change tracked via Tobii Eyetrackers. DS1 includes surgeons with low, medium, and high experience (by hours of RAS) as well as annotated tissue Tear (n=70) and Puncture (n=62) errors. DS2 contains data from low experience surgeons only and annotations for Tears (n=445) and Punctures (n=527). We extract 5 sec of video and varying history of ICA (from 5 to 60 sec) before and around human-labeled errors (positive label). We randomly sample the remaining data (negative label). We train variations of ML architecture consisting of video processing (pretrained Resnet18) and/or ICA processing (LSTM). We train the model on 3 different 80/20 train-test between-participant random splits. RESULTS: When including surgeons of mixed experience levels (DS1), ICA alone (AUC=64.28) is competitive with video alone (AUC=62.59) in detecting surgical errors (Figure 1A). However, the use of ICA in combination with video improves AUC in both datasets with gains of 5.55 (DS1) and 3.24 (DS2). History of ICA up to 20-30 seconds seems beneficial for error detection, after which point there is no further gain (Figure 1B). This is likely attributed to gradual build-up in CL leading to an error, and low-frequency of ICA measurements (1/sec), which may hinder extracting useful information from short time windows. The challenge of extracting meaningful information seems supported by Figure 1C, where providing processed ICA is beneficial (AUC=62.95) on top of raw ICA measurements (AUC=58.88). CONCLUSIONS: A surgeon’s CL in the context of surgical errors in RAS education is important. We show how CL is particularly valuable when a dataset includes mixed-experience surgeons. We also show the importance of ICA history length and pre-processing. Further work will explore better methods of extracting valuable features from CL data and its importance in surgery. Source of Funding: This material is based upon work supported by the National Science Foundation under Grant # 2030859 to the Computing Research Association for the CIFellows Project. Additionally, research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R01CA273031. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e831 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Rafal Kocielnik More articles by this author Jasper A. Laca More articles by this author Elyssa Y. Wong More articles by this author Timothy N. Chu More articles by this author Sidney I. Roberts More articles by this author Sandra Marshall More articles by this author Anima Anandkumar More articles by this author Andrew J. Hung More articles by this author Expand All Advertisement PDF downloadLoading ...