The influence of Non-Technical Skills of a Technical Skills Trainer (NTS-TeST) checklist on skills training: Perceptions from trainers and trainees

Surgical training has been greatly affected by the COVID-19 pandemic with limited opportunities and learning experiences for trainees in the operating theatre.1James H.K. Pattison G.T.R. Disruption to surgical training during covid-19 in the United States, United Kingdom, Canada, and australasia: a rapid review of impact and mitigation efforts.J Surg Educ. 2021; 78: 308-314Crossref PubMed Scopus (22) Google Scholar During recent years, there has also been a shift from time-based and numbers-based training to competency-based training, as demonstrated by the new ISCP curriculum.2ISCP surgical curriculum.2021https://www.iscp.ac.uk/iscp/curriculum-2021/Date accessed: September 19, 2022Google Scholar As such, more than ever, there is pressure for surgical trainers and trainees alike to maximise every learning experience. Simulation training is an effective training opportunity and has been increasingly used to fill the void created by the pandemic.3Doulias T. Gallo G. Rubio-Perez I. et al.Doing more with less: surgical training in the COVID-19 era.J Invest Surg. 2022; 35: 171-179Crossref PubMed Scopus (24) Google Scholar With the increased demand for simulation training from trainees, comes an increased need for confident and competent simulation trainers. The Non-Technical Skills of a Technical Skills Trainer (NTS-TeST) is a tool developed to aid in developing trainers’ operative teaching skills. It is an 18-point assessment tool with 4 subcategories: Pre-skill training, during skill training, post skill training and a global rating. These are scored out of 25, 40, 10 and 15 points, respectively, combined to give an overall score of 90. It is designed to evaluate trainers in a simulation context and allows trainers to reflect upon and improve their teaching skills.4Pears M. Biyani C.S. Joyce A.D. et al.Capturing the non-technical skills of a technical skills trainer (NTS-TeST) during simulation.Scot Med J. 2021; 66: 124-133Crossref PubMed Scopus (3) Google Scholar As such, this tool could be of significant utility to trainers and trainees alike if shown to . Cognitive Mental Load Theory is a framework within educational research that is used to evaluate learning environments.5Sweller J. Chandler P. Evidence for cognitive load theory.Cognit InStruct. 1991; 8: 351-362https://doi.org/10.1207/s1532690xci0804_5Crossref Scopus (253) Google Scholar It is based on the principle that when dealing with new information, working memory has a limited capacity and when this capacity is surpassed, learning is impaired. Recently, validated measurement tools have been developed to allow the measurement of cognitive load during simulation training. One such tool developed by Klepsch et al. is an 8-point assessment tool, scored out of 80.6Gonzalez-Vargas J.M. Tzamaras H.M. Martinez J. et al.Going the (social) distance: comparing the effectiveness of online versus in-person internal jugular central venous catheterization procedural training.Am J Surg. 2022 Sep; 224: 903-907Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar Here we assess the utility of NTS-TeST in the context of a formal surgical skills simulation session, to improve teaching quality for trainers and trainees alike and reduce the cognitive load on trainees. The study was conducted during the 6th Urology Simulation Boot Camp in Leeds, UK. It is a 5-day introductory hands-on course for the new urology registrars.7Young M. Kailavasan M. Taylor J. et al.The success and evolution of a urological "Boot camp" for newly appointed UK urology registrars: incorporating simulation, nontechnical skills and assessment.J Surg Educ. 2019 Sep-Oct; 76: 1425-1432Crossref PubMed Scopus (34) Google Scholar,8Young M. Lewis C. Kailavasan M. et al.A systematic review of methodological principles and delivery of surgical simulation bootcamps.Am J Surg. 2022 Jun; 223: 1079-1087Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar Twenty-four urology residents took part in a technical skill simulation session comprised of different stations supervised by 12 consultants (trainers) in the study. The first trainee at each station was asked to complete the NTS-TeST score and a Cognitive Mental Load score after completing the technical skills training session following the first encounter with a trainer. Trainers were not aware of this part. Each trainer also completed the NTS-TeST sheet (Supplementary File) and provided a self-rating of their first training episode. The second trainee at the same station with the same trainer again completed the NTS-TeST and a Cognitive Mental Load score after the training episode. Some of the teaching episodes were observed by expert trainers, and tips were shared. Trainers completed a self-rating of their teaching using the NTS-TeST as a checklist to improve teaching quality on their next station run. NTS-TeST scores for trainees and trainers pre- and post-checklist implementation were compared, and statistical significance was assessed using the unpaired t-test. The same was done for trainees’ Cognitive Mental Load scores. Pre- and post-checklist implementation trainers gave mean NTS-TeST self-ratings of 69.75 ± 2.54 and 77.50 ± 2.57 respectively (p = 0.0431). Pre- and post-checklist implementation Trainees gave mean NTS-TeST scores of 86.55 ± 0.87 and 86.58 ± 1.17 respectively (p = 0.9798). Pre- and post-checklist implementation trainees gave mean cognitive mental load scores of 42.18 ± 3.56 and 38.40 ± 2.42 respectively (p = 0.4002). These results are summarised in Table 1 below. Subcategory scores (Pre, During, Post, Global) for trainers' pre and post-checklist implementation and subcategory scores (Pre, During, Post, Global) for trainees’ pre and post-checklist implementation are summarised in Table 2.Table 1Trainers and trainees' NTS-TeST rating and trainees' cognitive mental load scores pre- and post-checklist implementation.Trainer NTS-TeST, Mean ( ± SE)Trainee NTS-TeST, Mean ( ± SE)Trainee Cognitive Mental Load, Mean ( ± SE)Pre-checklist69.75 (2.54)86.55 (0.87)42.18 (3.56)Post-checklist77.50 (2.57)86.58 (1.17)38.40 (2.42)Unpaired t- testp = 0.0431p = 0.9798P = 0.4002 Open table in a new tab Table 2Trainers’ and trainees, subcategory scores (Pre, During, Post, Global) pre- and post-checklist implementation.TrainersNTS-TeST Pre-skills training, Mean ( ± SE)NTS-TeST During skills training, Mean ( ± SE)NTS-TeST Post-skills training, Mean ( ± SE)NTS-TeST Global Rating, Mean ( ± SE)Pre-checklist17.00 (1.00)32.00 (1.09)8.08 (0.47)12.58 (0.56)Post-checklist20.67 (1.08)34.42 (1.26)9.08 (0.38)13.08(0.56)Unpaired t- testp = 0.0205p = 0.1613p = 0.1109p = 0.5320TraineesNTS-TeST Pre- skills training ( ± SE)NTS-TeST During- skills training ( ± SE)NTS-TeST Post- skills training ( ± SE)NTS-TeST Global Rating ( ± SE)Pre-Checklist23.91 (0.58)39.00 (0.73)9.91 (0.09)13.36 (0.58)Post-Checklist23.67 (0.57)38.75 (0.49)9.75 (0.18)14.42 (0.26)Unpaired t- testp = 0.7684p = 0.7755p = 0.4505p = 0.1012 Open table in a new tab Trainers noted an increase in the perceived quality of their teaching following the implementation of the checklist, as demonstrated by the increase in their NTS-TeST self-rating scores by 7.75. This represented an increase of 11.1% and was found to be statistically significant. When looking at the subcategories, the main area trainers felt they had improved was in the pre-skills training domain. As such, implementing the NTS-TeST as a checklist for trainers appears to be effective in increasing the trainer's perception of the quality of their teaching in the simulation setting, particularly in the pre-skill stage. This is clearly of use, as improving trainer confidence in simulation teaching will become increasingly important given the increased utilisation of simulation training within surgical teaching.4Pears M. Biyani C.S. Joyce A.D. et al.Capturing the non-technical skills of a technical skills trainer (NTS-TeST) during simulation.Scot Med J. 2021; 66: 124-133Crossref PubMed Scopus (3) Google Scholar,6Gonzalez-Vargas J.M. Tzamaras H.M. Martinez J. et al.Going the (social) distance: comparing the effectiveness of online versus in-person internal jugular central venous catheterization procedural training.Am J Surg. 2022 Sep; 224: 903-907Abstract Full Text Full Text PDF PubMed Scopus (1) Google Scholar However, this was not the case for trainees, with their perceived quality of teaching having no statistically significant difference despite the implementation of the checklist regardless of the NTS-TeST subcategory. As a result, whilst NTS-TeST seems to be a useful tool for trainers to reflect on their performance and make changes that they feel have improved it, trainees did not have the same perception. Interestingly, trainees gave very high overall NTS-TeST feedback scores before the checklist's implementation. This left little room for improvement with such a high initial score and demonstrated that trainees felt that the overall quality of the teaching was high both before and after checklist implementation. As to why trainees scored the teaching much higher than trainers in both pre-and post-checklist implementation, it may be that trainees were less objective about the teaching quality as they were pleased to be having the opportunity at all given the reduction in training experiences during the pandemic.7Young M. Kailavasan M. Taylor J. et al.The success and evolution of a urological "Boot camp" for newly appointed UK urology registrars: incorporating simulation, nontechnical skills and assessment.J Surg Educ. 2019 Sep-Oct; 76: 1425-1432Crossref PubMed Scopus (34) Google Scholar As a result, they rated the teaching highly regardless of its quality, while trainers who were more experienced were able to be more objective and therefore more critical of their teaching. It seems that, whilst utilising NTS-TeST as a checklist for trainers may be promising in reducing trainee cognitive mental load, this study has not definitively proven it. We acknowledge that only endoscopic and laparoscopic simulations were used to examine the two types of surgical skills. In addition, it is too early to assess whether the improvement noted by trainers was educationally significant. It would be beneficial to check in the future to determine if open skills simulation settings may yield the same encouraging results. It would also be valuable to see if this can be extended beyond the simulation setting and applied to a clinical setting, such as a training list in the operating theatre. The following is the Supplementary data to this article: Download .docx (.02 MB) Help with docx files Multimedia component 1