ObjectivesWorkplace-based assessment (WBA) is a key requirement of competency-based medical education in postgraduate surgical education. Although simulated workplace-based assessment (SWBA) has been proposed to complement WBA, it is insufficiently adopted in surgical education. In particular, approaches to criterion-referenced and automated assessment of intraoperative surgical competency in contextualized SWBA settings are missing.Main objectives were (1) application of the universal framework of intraoperative performance and exemplary adaptation to spine surgery (vertebroplasty); (2) development of computer-assisted assessment based on criterion-referenced metrics; and (3) implementation in contextualized, team-based operating room (OR) simulation, and evaluation of validity.DesignMultistage development and assessment study: (1) expert-based definition of performance indicators based on framework’s performance domains; (2) development of respective assessment metrics based on preoperative planning and intraoperative performance data; (3) implementation in mixed-reality OR simulation and assessment of surgeons operating in a confederate team. Statistical analyses included internal consistency and interdomain associations, correlations with experience, and technical and non-technical performances.SettingSurgical simulation center. Full surgical team set-up within mixed-reality OR simulation.ParticipantsEleven surgeons were recruited from two teaching hospitals. Eligibility criteria included surgical specialists in orthopedic, trauma, or neurosurgery with prior VP or kyphoplasty experience.Main outcome measuresComputer-assisted assessment of surgeons’ intraoperative performance.ResultsPerformance scores were associated with surgeons’ experience, observational assessment (Objective Structured Assessment of Technical Skill) scores and overall pass/fail ratings. Results provide strong evidence for validity of our computer-assisted SWBA approach. Diverse indicators of surgeons’ technical and non-technical performances could be quantified and captured.ConclusionsThis study is the first to investigate computer-assisted assessment based on a competency framework in authentic, contextualized team-based OR simulation. Our approach discriminates surgical competency across the domains of intraoperative performance. It advances previous automated assessment based on the use of current surgical simulators in decontextualized settings. Our findings inform future use of computer-assisted multidomain competency assessments of surgeons using SWBA approaches.

Flow disruptions (FD) in the operating room (OR) have been found to adversely affect the levels of stress and cognitive workload of the surgical team. It has been concluded that frequent disruptions also lead to impaired technical performance and subsequently pose a risk to patient safety. However, respective studies are scarce. We therefore aimed to determine if surgical performance failures increase after disruptive events during a complete surgical intervention. We set up a mixed-reality-based OR simulation study within a full-team scenario. Eleven orthopaedic surgeons performed a vertebroplasty procedure from incision to closure. Simulations were audio- and videotaped and key surgical instrument movements were automatically tracked to determine performance failures, i.e. injury of critical tissue. Flow disruptions were identified through retrospective video observation and evaluated according to duration, severity, source, and initiation. We applied a multilevel binary logistic regression model to determine the relationship between FDs and technical performance failures. For this purpose, we compared FDs in one-minute intervals before performance failures with intervals without subsequent performance failures. Average simulation duration was 30:02 min (SD = 10:48 min). In 11 simulated cases, 114 flow disruption events were observed with a mean hourly rate of 20.4 (SD = 5.6) and substantial variation across FD sources. Overall, 53 performance failures were recorded. We observed no relationship between FDs and likelihood of immediate performance failures: Adjusted odds ratio = 1.03 (95% CI 0.46–2.30). Likewise, no evidence could be found for different source types of FDs. Our study advances previous methodological approaches through the utilisation of a mixed-reality simulation environment, automated surgical performance assessments, and expert-rated observations of FD events. Our data do not support the common assumption that FDs adversely affect technical performance. Yet, future studies should focus on the determining factors, mechanisms, and dynamics underlying our findings.

Introduction We present a novel 3-dimensional (3D) printing method for low-cost and widely available reproduction of computed tomography (CT)-based synthetic bone models for spine surgery simulation, optimized to reproduce realistic haptic properties. The method allows reproduction of either normal or abnormal patient anatomy. The models are fluoroscopy compatible and contain deformities and fractures present in the underlying CT data. Methods Spine models created from CT data were printed on a 3D printer using 2 different materials for cortical and cancellous bone. Printing parameters were iteratively optimized with surgical experts and 3 candidate spine models were evaluated in a study regarding haptic properties. X-ray images of a spine section printed with final printing parameters were evaluated by surgical experts regarding fluoroscopic properties. Results Eleven surgical experts performed a trocar insertion, a typical workflow step in spine surgery procedures, on the models. We observed agreement that cortical structures and strong agreement that cancellous structures of the final model are haptically comparable with human vertebral bone. Ten surgical experts evaluated x-ray images of the model. They expressed strong agreement on the similarity with x-ray images of the human spine and confirmed the presence of a fracture. Material cost of a typical spine model is around US $11. Conclusions Models created using the novel methodology realistically reproduce the haptic properties during a trocar placement into the vertebral body. The models are compatible with conventional x-ray imaging. Because the models correspond to real patient CT data, those can alternatively be used in simulation environments that simulate fluoroscopy or CT image guidance to produce highly realistic, radiation-free imaging output.

scores (t1⁄40.60; P1⁄4 0.007). This association was attenuated Study Design. Observational simulation study. Objective. The goal of this study was to investigate the relationship between technical and nontechnical skills (NTS) in a simulated surgical procedure. Summary of Background Data. Although surgeons’ technical and NTS during surgery are crucial determinants for clinical outcomes, little literature is available in spine surgery. Moreover, evidence regarding how surgeons’ technical and NTS are related is limited. Methods. A mixed-reality and full-scale simulated operating room environment was employed for the surgical team. Eleven surgeons performed the vertebroplasty procedure (VP). Technical skills (TS) were assessed using Objective Structured Assessment of Technical Skill scores and senior expert-evaluated VP outcome assessment. NTS were assessed with the Observational Teamwork Assessment for Surgery. Kendall-Tau-b tests were performed for correlations. We further controlled the influence of surgeons0 experience (based on professional tenure and number of previous VPs performed). Result. Surgeons’ NTS correlated significantly with their technical performance (t1⁄40.63; P1⁄40.006) and surgical outcome

Background. Virtual reality (VR)-based simulations offer rich opportunities for surgical skill training and assessment of surgical novices and experts. A structured evaluation and validation process of such training and assessment tools is necessary for effective surgical learning environments. Objective. To develop and apply a classification system of surgeon-reported experience during operation of a VR vertebroplasty simulator. Methods. A group of orthopedic, trauma surgeons and neurosurgeons (n = 13) with various levels of expertise performed on a VR vertebroplasty simulator. We established a mixed-methods design using think-aloud protocols, senior surgical expert evaluations, performance metrics, and a post-simulation questionnaire. Verbal content was systematically analyzed using structured qualitative content analysis. We established a category system for classification of surgeons' verbal evaluations during the simulation. Furthermore, we evaluated intraoperative performance metrics and explored potential associations with surgeons' characteristics and simulator evaluation. Results. Overall, 244 comments on realism and usability of the vertebroplasty simulator were collected. This included positive and negative remarks, questions, and specific suggestions for improvement. Further findings included surgeons' approval of the realism and usability of the simulator and the observation that the haptic feedback of the VR patient's anatomy requires further improvement. Surgeon-reported evaluations were not associated with performance decrements. Discussion. This study is the first to apply think-aloud protocols for evaluation of a surgical VR-based simulator. A novel classification approach is introduced that can be used to classify surgeons' verbalized experiences during simulator use. Our lessons learned may be valuable for future research with similar methodological approach.

Study Design. Observational simulation study. Objective. The goal of this study was to investigate the relationship between technical and nontechnical skills (NTS) in a simulated surgical procedure. Summary of Background Data. Although surgeons' technical and NTS during surgery are crucial determinants for clinical outcomes, little literature is available in spine surgery. Moreover, evidence regarding how surgeons' technical and NTS are related is limited. Methods. A mixed-reality and full-scale simulated operating room environment was employed for the surgical team. Eleven surgeons performed the vertebroplasty procedure (VP). Technical skills (TS) were assessed using Objective Structured Assessment of Technical Skill scores and senior expert-evaluated VP outcome assessment. NTS were assessed with the Observational Teamwork Assessment for Surgery. Kendall-Tau-b tests were performed for correlations. We further controlled the influence of surgeons' experience (based on professional tenure and number of previous VPs performed). Result. Surgeons' NTS correlated significantly with their technical performance (tau = 0.63; P = 0.006) and surgical outcome scores (tau = 0.60; P = 0.007). This association was attenuated when controlling for surgeons' experience. Conclusion. Our results suggest that spine surgeons with higher levels of TS also apply better communication, leadership, and coordination behaviors during the procedure. Yet, the role of surgeons' experience needs further investigation for improving surgeons' intraoperative performance during spine surgery.

We share our experiences in designing a complete simulator prototype and provide the technological basis to determine whether an immersive medical training environment for vertebroplasty is successful. In our study, the following key research contributions were realised: (1) the effective combination of a virtual reality surgical simulator and a computerised mannequin in designing a novel training setup for medical education, and (2) based on a user-study, the quantitative evaluation through surgical workflow and crisis simulation in proving the face validity of our immersive medical training environment.Medical simulation platforms intend to assist and support surgical trainees by enhancing their skills in a virtual environment. This approach to training is consistent with an important paradigm shift in medical education that has occurred over the past decade. Surgical trainees have traditionally learned interventions on patients under the supervision of a senior physician in what is essentially an appre...

PurposeThe discrepancy of continuously decreasing opportunities for clinical training and assessment and the increasing complexity of interventions in surgery has led to the development of different training and assessment options like anatomical models, computer-based simulators or cadaver trainings. However, trainees, following training, assessment and ultimately performing patient treatment, still face a steep learning curve.MethodsTo address this problem for C-arm-based surgery, we introduce a realistic radiation-free simulation system that combines patient-based 3D printed anatomy and simulated X-ray imaging using a physical C-arm. To explore the fidelity and usefulness of the proposed mixed-reality system for training and assessment, we conducted a user study with six surgical experts performing a facet joint injection on the simulator.ResultsIn a technical evaluation, we show that our system simulates X-ray images accurately with an RMSE of 1.85mm compared to real X-ray imaging. The participants expressed agreement with the overall realism of the simulation, the usefulness of the system for assessment and strong agreement with the usefulness of such a mixed-reality system for training of novices and experts. In a quantitative analysis, we furthermore evaluated the suitability of the system for the assessment of surgical skills and gather preliminary evidence for validity.ConclusionThe proposed mixed-reality simulation system facilitates a transition to C-arm-based surgery and has the potential to complement or even replace large parts of cadaver training, to provide a safe assessment environment and to reduce the risk for errors when proceeding to patient treatment. We propose an assessment concept and outline the steps necessary to expand the system into a test instrument that provides reliable and justified assessments scores indicative of surgical proficiency with sufficient evidence for validity.

Background Despite the growing importance of medical simulation in education, there is limited guidance available on how to develop medical simulation environments, particularly with regard to technical and non-technical skills as well as to multidisciplinary operating room (OR) team training. We introduce a cognitive task analysis (CTA) approach consisting of interviews, structured observations, and expert consensus to systematically elicit information for medical simulator development. Specifically, our objective was to introduce a guideline for development and application of a modified CTA to obtain task demands of surgical procedures for all three OR professions with comprehensive definitions of OR teams’ technical and non-technical skills. Methods To demonstrate our methodological approach, we applied it in vertebroplasty, a minimally invasive spine procedure. We used a CTA consisting of document reviews, in situ OR observations, expert interviews, and an expert consensus panel. Interviews included five surgeons, four OR nurses, and four anesthetists. Ten procedures were observed. Data collection was carried out in five OR theaters in Germany. Results After compiling data from interviews and observations, we identified 6 procedural steps with 21 sub-steps for surgeons, 20 sub-steps for nurses, and 22 sub-steps for anesthetists. Additionally, we obtained information on 16 predefined categories of intra-operative skills and requirements for all three OR professions. Finally, simulation requirements for intra-operative demands were derived and specified in the expert panel. Conclusions Our CTA approach is a feasible and effective way to elicit information on intra-operative demands and to define requirements of medical team simulation. Our approach contributes as a guideline to future endeavors developing simulation training of technical and non-technical skills for multidisciplinary OR teams.

Background Context Virtual reality (VR)-based simulators offer numerous benefits and are very useful in assessing and training surgical skills. Virtual reality-based simulators are standard in some surgical subspecialties, but their actual use in spinal surgery remains unclear. Currently, only technical reviews of VR-based simulators are available for spinal surgery. Purpose Thus, we performed a systematic review that examined the existing research on VR-based simulators in spinal procedures. We also assessed the quality of current studies evaluating VR-based training in spinal surgery. Moreover, we wanted to provide a guide for future studies evaluating VR-based simulators in this field. Study Design and Setting This is a systematic review of the current scientific literature regarding VR-based simulation in spinal surgery. Methods Five data sources were systematically searched to identify relevant peer-reviewed articles regarding virtual, mixed, or augmented reality-based simulators in spinal surgery. A qualitative data synthesis was performed with particular attention to evaluation approaches and outcomes. Additionally, all included studies were appraised for their quality using the Medical Education Research Study Quality Instrument (MERSQI) tool. Results The initial review identified 476 abstracts and 63 full texts were then assessed by two reviewers. Finally, 19 studies that examined simulators for the following procedures were selected: pedicle screw placement, vertebroplasty, posterior cervical laminectomy and foraminotomy, lumbar puncture, facet joint injection, and spinal needle insertion and placement. These studies had a low-to-medium methodological quality with a MERSQI mean score of 11.47 out of 18 (standard deviation=1.81). Conclusions This review described the current state and applications of VR-based simulator training and assessment approaches in spinal procedures. Limitations, strengths, and future advancements of VR-based simulators for training and assessment in spinal surgery were explored. Higher-quality studies with patient-related outcome measures are needed. To establish further adaptation of VR-based simulators in spinal surgery, future evaluations need to improve the study quality, apply long-term study designs, and examine non-technical skills, as well as multidisciplinary team training.

The discrepancy of continuously decreasing clinical training opportunities and increasing complexity of interventions in surgery has led to the development of different training options like anatomical models, computer-based simulators or cadaver trainings. However, trainees, following this training and ultimately performing patient treatment, still face a steep learning curve. To address this problem for C-arm based surgery, we introduce a realistic radiation-free simulation system that combines patient-based 3D printed anatomy and simulated X-ray imaging using a physical C-arm. This mixed reality simulation system facilitates a transition to C-arm based surgery and has the potential to complement or even replace large parts of cadaver training and to reduce the risk for errors when proceeding to patient treatment. In a technical evaluation, we show that our system simulates X-ray images accurately with an RMSE of 1.85 mm compared to real X-ray imaging. To explore the fidelity and usefulness of the proposed mixed reality system for training and assessment, we conducted a user study. Six surgical experts performed a facet joint injection on the simulator and rated aspects of the system on a 5-point Likert scale. They expressed agreement with the overall realism of the simulation and strong agreement with the usefulness of such a mixed reality system for training of novices and experts.

BACKGROUND AND AIM:Surgical flow disruptions occur frequently and jeopardize perioperative care and surgical performance. So far, insights into subjective and cognitive implications of intra-operative disruptions for surgeons and inherent consequences for performance are inconsistent. This study aimed to investigate the effect of surgical flow disruption on surgeon's intra-operative workload and technical performance.METHODS:In a full-scale OR simulation, 19 surgeons were randomly allocated to either of the two disruption scenarios (telephone call vs. patient discomfort). Using a mixed virtual reality simulator with a computerized, high-fidelity mannequin, all surgeons were trained in performing a vertebroplasty procedure and subsequently performed such a procedure under experimental conditions. Standardized measures on subjective workload and technical performance (trocar positioning deviation from expert-defined standard, number, and duration of X-ray acquisitions) were collected.RESULTS:Intra-operative workload during simulated disruption scenarios was significantly higher compared to training sessions (p < .01). Surgeons in the telephone call scenario experienced significantly more distraction compared to their colleagues in the patient discomfort scenario (p < .05). However, workload tended to be increased in surgeons who coped with distractions due to patient discomfort. Technical performance was not significantly different between both disruption scenarios. We found a significant association between surgeons' intra-operative workload and technical performance such that surgeons with increased mental workload tended to perform worse (β = .55, p = .04).CONCLUSIONS:Surgical flow disruptions affect surgeons' intra-operative workload. Increased mental workload was associated with inferior technical performance. Our simulation-based findings emphasize the need to establish smooth surgical flow which is characterized by a low level of process deviations and disruptions.

Stereo-tactic neurosurgery planning is a time-consuming and complex task that requires detailed understanding of the patient anatomy and the affected regions in the brain to precisely deliver the treatment and to avoid proximity to any known risk structures. Traditional user interfaces for neurosurgery planning use keyboard and mouse for interaction and visualize the medical data on a screen. Previous research, however, has shown that 3D user interfaces are more intuitive for navigating volumetric data and enable users to understand spatial relations more quickly. Furthermore, new imaging modalities and automated segmentation of relevant structures provide important information to medical experts. However, displaying such information requires frequent context switches or occludes otherwise important information. In collaboration with medical experts, we analyzed the planning workflow for stereo-tactic neurosurgery interventions and identified two tasks in the process that can be improved: volume exploration and trajectory refinement. In this paper, we present a novel 3D user interface for neurosurgery planning that is implemented using a head-mounted display and a haptic device. The proposed system improves volume exploration with bi-manual interaction to control oblique slicing of volumetric data and reduces visual clutter with the help of haptic guides that enable users to precisely target regions of interest and to avoid proximity to known risk structures.

We present a unique simulator-based methodology for assessing both technical and nontechnical (cognitive) skills for surgical trainees while immersed in a complete medical simulation environment. Further, we have included two crisis scenarios which allow for the evaluation of the effect of cognitive strategy selection on the low-level surgical skills. Training these mixed-mode scenarios can thereby be evaluated on our platform, allowing for improved assessment and a stronger foundation for credentialing, with the potential to reduce the occurrence of adverse events in the operating room. Scientific evaluation and validation of our work is conducted together with 19 junior surgeons in order to achieve the following goals: 1) to provide a qualitative measure of usability, 2) to assess vertebroplasty technical performance of the surgeon, and 3) to explore the relationship between mental workload and surgical performance during crisis. Our results indicate that: 1) the surgeons scored the face validity of our modeled simulation environment very highly ( 4.68 ±0.48, using a 5-point Likert scale), 2) surgeon training enabled completion of tasks more quickly, and 3) the introduction of crisis scenarios negatively affected the surgeons' objective performance. Taken together, our results underscore the need to develop realistic simulation environments that prepare young residents to respond to emergent events in the operating room.

We present a prototype of a system in development for pre-operative planning. The proposed NeuroTable uses a combination of traditional rendering and novel visualization techniques rendered to facilitate real-time collaboration between neurosurgeons during intervention planning. A set of multimodal 2D and 3D renderings convey the relation between the region of interest and the surrounding anatomical structures. A haptic device is used for interaction with the NeuroTable to facilitate immersive control over the 3D cursor and navigation modes for the neurosurgeons during their discourse of planning. A pilot experimental study was conducted to assess the performance of users in targeting points within the preoperative 3D scan. Then, two clinicians participated in the evaluation of the table in discussing and planning a case. Results indicate that the NeuroTable facilitated the discourse and we discuss the results of the speed and accuracy for the specification of entry and target points.

We present a medical Augmented Reality (AR) edutainment system for bone anatomy learning. This learning environment, called AR bone puzzle, is a metaphor for bone anatomy learning with AR visualization and intuitive interaction. AR bone puzzle uses its user's body as a puzzle frame and computer generated virtual bones as puzzle pieces. Users learn bone anatomy by assembling the virtual bone pieces on their body. Key features of this system are 3D AR visualization and intuitive gesture based user interaction.

PURPOSE:To design a surgical training environment based on task and crisis analysis of the surgical workflow.METHOD:The environment consists of: (1) real surgical instruments that are augmented with realistic haptic feedback and VR capabilities, (2) human sensory channels such as tactile, auditory and visual in real time, and (3) the ability to facilitate deliberate exposure to adverse events enabling mediation of error recovery strategies.VALIDATION:Five surgeons were immersed in our medical simulation environment through task and crisis scenarios of a typical vertebroplasty workflow.RESULTS:Based on a five-point Likert-scale survey, the face validity of our simulation environment was confirmed by investigating surgeon behavior and workflow response.CONCLUSIONS:The result of the conducted user-study corroborates our unique medical simulation concept of combining VR and human multisensory responses into surgical workflow.

We present the idea of a user interface concept, which resolves the challenges involved in the control of angiographic C-arms for their constant repositioning during interventions by either the surgeons or the surgical staff. Our aim is to shift the paradigm of interventional image acquisition workflow from the traditional control device interfaces to ‘desired-view’ control. This allows the physicians to only communicate the desired outcome of imaging, based on simulated X-rays from pre-operative CT or CTA data, while the system takes care of computing the positioning of the imaging device relative to the patient’s anatomy through inverse kinematics and CT to patient registration. Together with our clinical partners, we evaluate the new technique using 5 patient CTA and their corresponding intraoperative X-ray angiography datasets.

We present a method in designing a medical simulation environment based on task and crisis analysis of the surgical workflow. The environment consists of real surgical tools and instruments that are augmented with realistic haptic feedback and VR capabilities. Inherently, we also addressed a broad spectrum of human sensory channels such as tactile, auditory and visual in real-time. Lastly, the proposed approach provides a simulation environment facilitating deliberate exposure to adverse events enabling mediation of error recovery strategies. To validate the face validity of our simulator design we chose a spinal procedure, the vertebroplasty, in which four expert surgeons were immersed in our medical simulation environment. Based on a Likert-scale questionnaire, the face validity of our simulation environment was assessed by investigating surgeon behavior and workflow response. The result of the conducted user-study corroborates our unique medical simulation concept of combining VR and human multisensory responses into surgical workflow.