INTRODUCTION:Identifying acuity and patient prioritization are essential but difficult skills to teach to medical students. This study aimed to develop and assess the initial validity evidence of a checklist to be used in simulation scenarios for fourth-year medical students addressing these skills. METHODS:We applied the Kane validation framework to gather evidence supporting the use of this checklist in simulation-based assessment, focusing on the Scoring and Generalization domains. A modified Delphi method was used to create an ideal performance checklist to be used in simulation scenarios of patients requiring urgent or emergent care. Following development, a Mastery Angoff approach was used to establish a Minimum Passing Standard (MPS). Fourth-year medical students were recruited to participate in simulations, and performance was assessed using the created checklists. During the simulations, students received patient handoffs and were asked to prioritize acuity and manage evolving clinical situations across 2 simulated encounters. Simulations were scored by both real-time and video-observed raters and inter-rater reliability (IRR) was calculated. RESULTS:Twenty-five students participated in the 2023 to 2024 academic year. The expert panel finalized a 10-item checklist and the MPS was determined to be 85%. Overall, IRR was excellent, with a Cohen kappa coefficient of 0.872. Cohen kappa coefficient for individualized checklist domains ranged from 0.65 to 1. CONCLUSION:This study describes the development of a performance checklist assessing students' abilities in identifying acuity and patient prioritization, with initial validity evidence supporting Kane Scoring and Generalization inferences.
INTRODUCTION:The creation of simulation scenarios is typically time-consuming and requires a high cognitive load. Artificial intelligence (AI) and large language models (LLMs) can generate text, including simulation scenarios, but the output quality can be very user-dependent. The goal of this study is to explore if a customized version of ChatGPT can create simulation scenarios that are comparable to those written by humans. METHODS:In this observational cross-sectional study, simulation content experts scored 8 scenarios covering 4 pediatric emergency medicine (PEM) topics with one version written by health care educators and published on MedEdPORTAL, and another by a custom ChatGPT. Experts were blinded to the source of each case. The scenarios were assessed in 3 ways: an objective score from an established simulation scoring checklist, a subjective quality rating, and whether the experts would recommend the scenario for use. RESULTS:Seven experts with an average of 11 years of PEM training and an average of 10 years of simulation work scored simulations. While all 4 AI-generated cases had lower average scores on both quantitative assessments than their human-written counterparts, these results were not statistically significant. The majority of experts recommended each case to colleagues, regardless of source. There was moderate to excellent intrarater reliability. CONCLUSION:Human-written cases received higher scores than the AI-written cases. While these scores were not statistically significant, these differences highlight the importance of keeping a human-expert influence when using AI to produce simulation cases. This ChatGPT tool can therefore be useful in assisting in the timely creation of scenarios, allowing facilitators, regardless of their technical skill with AI, to decrease cognitive load. Next steps should move beyond evaluating the scenario text to include evaluation of the experience of facilitating the ChatGPT-derived pediatric simulations as well as assessing the perspectives of simulation operators.
Intro: Simulated patients (SP) are individuals trained to portray patients in clinical encounters for health care education. This study explores potential health benefits and knowledge resulting from SP work. Methods: Ten SPs at the researchers’ institution were invited to 1-on-1 semistructured interviews that were recorded and transcribed. The authors approached the study through the Interpretative Phenomenological Analysis (IPA) framework to explore how SPs make sense of their experiences and the meanings they attach to them. Themes and sub-themes were developed from participant experiences and reflected upon by the authors. Results: Three major themes emerged: Increased knowledge of the way medicine works, taking action, and personal and interpersonal well-being. SPs found that their knowledge of medical terms, healthy behaviors, and disease risks increased and resulted in changes to their lifestyle and experience in medical settings. They also noted a sense of purpose in their work, its intellectual rigor, and a newfound community, which made it a positive experience. The findings of this study align with models of health behavior while reinforcing and challenging prior qualitative studies of SPs. Conclusion: SPs can and do learn and apply knowledge obtained from their patient portrayals. They are also personally enriched by their role. Further analysis should examine quantitative health impacts and the specific experiences or scenarios that lead to changes in SPs' lifestyles and health behaviors. Community organizations may find simulated patient training and encounters to be a useful tool for teaching healthy behaviors and communication with physicians.
Statement: Simulation is increasingly used not only to train clinicians but also to explore how clinical systems function in practice. As simulation becomes embedded in everyday health care settings, it is shaped by the conditions of clinical work. This raises questions about how simulation should be understood once it becomes part of the systems it is intended to influence. Drawing on concepts from complexity science and implementation research, this article examines simulation as a complex intervention introduced into complex health care systems. Rather than focusing on how simulation is designed or implemented, it offers a conceptual perspective for interpreting how simulation changes after becoming embedded in clinical practice. From this viewpoint, differences in how simulation sessions unfold are not necessarily signs of poor implementation but may instead reveal how simulation interacts with local clinical conditions.
INTRODUCTION:Clear and accurate handover from paramedics to emergency department staff is crucial to patient outcomes, but mental workload experienced throughout an emergency response may impair paramedics' ability to retain key clinical details for handover. This pilot study used a mixed factorial design to examine whether mental workload experienced throughout a simulated emergency response and clinical experience influences handover performance and physiological responses in paramedics and paramedicine students. METHODS:Ten paramedicine students and 6 experienced paramedics completed 2 full-scale simulated cases from dispatch, response drive, patient care, patient transport, and handover, under high and low mental workload conditions. Participants delivered verbal handovers at the conclusion of each case. Heart rate and heart rate variability were recorded across all phases as exploratory measures of physiological arousal. Subjective workload and handover performance were assessed after each scenario. RESULTS:A significant workload by experience interaction was observed for handover performance, although follow-up comparisons were not statistically significant. Descriptively, students recalled fewer case-relevant details at handover after a high-workload case, whereas experienced paramedics appeared unaffected, and a moderate effect size was observed for students. Physiological data indicated greater autonomic arousal during patient care than other phases; however, movement and postural differences across phases may have contributed to this pattern, limiting attribution to the mental workload manipulation. CONCLUSIONS:These preliminary findings highlight the need for further research with larger samples and suggest paramedicine training may benefit from supporting students in developing strategies to manage mental workload and retain key information during high-demand emergency responses.
This study explores the role of haptic feedback in dental virtual simulations by examining its effects on learner performance, usability, and self-confidence across novice and advanced students. Using a randomized crossover design, participants engaged with an implantology-related virtual simulation using Virteasy Dental software and Haply Robotics’ haptic arm in both haptic-enabled and haptic-disabled conditions. Advanced students significantly outperformed novices, supporting construct validity, though all other performance effects were nonsignificant. Usability and self-confidence were significantly higher for students who experienced the haptic-enabled simulation after first using the haptic-disabled version, indicating an order effect. However, no other usability or self-confidence interactions reached significance. Agreement between expert-generated and software-generated scores was moderate to poor, highlighting limitations in automated scoring and the need for expert oversight during student assessment. Findings show the value of portable haptics and call for better validated assessment and usability tools, and that trainees may benefit from first trying haptic-disabled virtual simulations before learning with haptics.
This manuscript focuses on mistakes and missteps that frequently occur in simulation-based learning. Using vignettes, problematic practices are highlighted and reviewed using current simulation standards and literature. Gaps in the literature are also addressed. Acknowledging and improving practices in simulation enhance learner outcomes, support novice educators, and assist individuals preparing for certification exams. Organizations that avoid deficiencies in simulation design and implementation, such as those presented in these vignettes, better align their simulation programs with core accreditation standards.
Introduction: Traditional simulation-based communication training remains resource-intensive and difficult to scale. While artificial intelligence (AI), particularly large language models, offers promising solutions for health care education, no blueprint exists for integrating AI-powered simulation training within operational quality improvement (QI) frameworks. This paper presents a 5-phase methodological blueprint, with evaluatory evidence, for implementing AI-powered simulation training to enhance transitional care communication skills. Methods: We developed a 5-phase methodological framework integrating AI simulation with QI and educational principles, grounded in the Donabedian model. The phases comprised: (1) content validation using Lawshe’s methodology, (2) simulation development applying cognitive load theory, (3) platform selection through expert consensus evaluation, (4) structured implementation including structural foundations and sequential deployment, and (5) outcome measurement using statistical process control. Results: Iterative testing across multiple AI platforms revealed that traditional debriefing approaches (advocacy-inquiry, plus-delta) could not be reliably delivered by AI systems. Microdebriefing with rubric-focused feedback emerged as optimal for AI-mediated learning, leveraging AI’s strengths in consistent, structured feedback delivery while working within limitations in complex facilitation. Clinical outcomes derived from rubric-based evaluations of recorded patient calls will be reported separately to allow for a more detailed analysis of communication quality and training effectiveness, which falls outside the scope of the present methods-focused study. Conclusions: This 5-phase methodological blueprint provides an approach for health care organizations seeking to implement cost-effective, scalable AI-powered communication training. By embedding simulation within QI infrastructure, institutions can systematically enhance communication skills while maintaining educational rigor. This work contributes to simulation science, telehealth education, and QI by demonstrating how AI can serve as a scalable alternative to traditional facilitator-led training for foundational communication skills.
Healthcare simulation centers collect extensive operational, educational, and experiential data, yet much of this information is often underutilized beyond accreditation reporting. The authors introduce the CORE Analytics Framework—Compliance, Operations, Results, and Experience—as a structured approach to transform routine simulation data into actionable operational intelligence. Rather than focusing on learner-facing pedagogical applications of artificial intelligence (AI), this framework targets administrative and logistical functions that influence educational quality and sustainability. By integrating correlation analysis, predictive analytics, and AI-assisted data synthesis, the CORE framework helps simulation leaders identify inefficiencies, anticipate resource needs, and support continuous improvement. A case example illustrates how scalable implementation can improve outcomes and optimize resource utilization, positioning operational intelligence as foundational to educational excellence in healthcare simulation.
To address a critical gap in trauma-informed de-escalation training for interprofessional pediatric staff and trainees, a virtual reality (VR) simulation program was developed and implemented at a Canadian tertiary care hospital. This early-stage evaluation assessed the feasibility, acceptability, and usability of immersive VR as a psychologically safe and engaging learning modality as part of a formally approved quality improvement initiative. Developed through multidisciplinary collaboration, the simulation integrated trauma-informed pedagogy within a realistic, emotionally complex scenario in which staff practiced responding to escalating patient distress. The mixed-method evaluation included posttraining surveys, facilitated debriefs, and learner-generated summaries guided by the New World Kirkpatrick Model. Participants from diverse patient-facing roles reported high engagement, psychological safety, and commitment to applying learned skills. Findings suggest trauma-informed VR simulation can support early indicators of capacity building across cognitive (decision-making), affective (emotional awareness and confidence), and interpersonal (communication and collaboration) learning domains. Immersive VR simulation shows promise as an effective learning strategy for preparing pediatric health care teams for complex behavioral escalation scenarios.
Simulation training in patient, family, and caregiver education has gained attention from the health care simulation community more recently. This scoping review aimed to describe the current state of the science surrounding the application of simulation training in patient, family, and caregiver education in comparison to more traditional learners like health professionals and their trainees. The final review included 41 articles describing the application of simulation training in both adult and pediatric patient populations across a variety of care settings. The opportunities to use simulation training in patient and caregiver education are vast; however, standards of best practice to guide the use of simulation in this unique population are currently lacking.
Entrustable professional activities (EPAs) transform abstract clinical competencies into practical measurable tasks. Integrating simulation and EPAs may support continuous improvement in health professions training by linking competency development and observable clinical tasks in a standardized environment. This study analyses the usability of EPAs in simulation-based education. The review search was conducted in 6 databases up to May 27, 2025. Twenty-two studies, including 3144 health care trainees, were included. EPA-integrated simulations improved learners’ confidence across different stages of health care training. EPA-based evaluations in simulation also yield moderately to highly reliable scores and show emerging validity with positive correlation with other competence measures. Important practical barriers include high resource requirements, rater calibration, and assessment challenges. EPAs in simulation-based health care education, especially up to the graduate level, may be valid tools to support learning and feedback. Preliminary evidence suggests potential utility for competency-based progression, but further research is needed to define a resource-sensitive implementation.
Introduction: Manikin-based high-fidelity simulation (HFS) is critical in anesthesia for teaching crisis resource management skills in a safe environment. However, HFS is known to result in significant stress, potentially affecting learning. Virtual reality (VR) has emerged as a promising alternative to HFS, but the impact on stress remains unknown. This study compares perceived and physiological stress in anesthesia trainees during VR and HFS, and in high- and low-performing subjects after HFS. Methods: This secondary analysis from a prospective, sequential, randomized controlled trial compares anesthesia trainee volunteers who undergo either VR or HFS scenarios. Perceived stress was measured using the State-Trait Anxiety Inventory (STAI), the Medical Emotion Scale (MES), and the NASA Task Load Index (NASA-TLX). Physiological stress was assessed via heart rate variability standard deviation between successive N-N intervals (SDNN) and the low-frequency/high-frequency (LF/HF) ratio. Trainees’ performance was assessed with the Ottawa Global Rating Scale. Results: Among the 34 volunteers enrolled, both the VR and HFS groups demonstrated similar STAI, MES, and NASA-TLX scores following the scenarios. However, VR elicited lower SDNN values (mean difference, −12, 95% CI, −23 to −1.0, P =0.034), but not the LF/HF ratio. No difference was observed in stress indicators between high-performing and low-performing trainees. Conclusions: Anesthesia trainees experience comparable levels of perceived stress in the form of anxiety, emotional responses, and task load in VR and HFS scenarios. A lower heart rate variability during VR scenarios may indicate greater physiological stress. The relevance and implications of these findings in simulation require further clarification. Study Registration ClinicalTrials.gov (NCT05041049).
Simulation is essential for health professions education, but a rising demand for simulation operations personnel strains an already overextended workforce. In response, we are developing a competency-based minor pathway within an undergraduate degree designed to train students without clinical backgrounds as nonclinical simulation operations specialists (SOS). This work exposed challenges in competency-based education, where assessing competence and determining readiness for autonomous professional practice remain difficult. To strengthen competency assessment and ensure workforce readiness, we propose scaling entrustable professional activities (EPAs), originally designed for medical education, to nonclinical SOS training. To accomplish this, we propose a 4-phase research process guided by the Medical Research Council framework for the design of complex interventions: (1) theory and evidence identification; (2) framework modeling; (3) piloting; and (4) hybrid evaluation and implementation. We anticipate that the resulting EPA-based assessment framework will hold the potential to enable rigorous, defensible assessment in nonclinical SOS training, providing a replicable model for future EPA-based program development in emerging competency-based nonclinical professional fields.
Purpose: This study aimed to evaluate the training outcomes and early transfer of ultrasound to the clinical environment of a simulation-based longitudinal ultrasound training program for critical care physicians in a single-center intensive care unit. Materials and Methods: We designed a simulation-based longitudinal ultrasound training for emergency and intensive care residents. The course lasts for 6 months per session, including theoretical classes, practice with simulators and real human models, learning of critical care ultrasound processes and protocols, and finally clinical application. A total of 164 physicians have received training through this program. We evaluated learner satisfaction, theoretical knowledge and practical performance both before and after training, as well as process-based indicators of clinical application and transfer of ultrasound skills into the clinical environment. Results: Across all 10 survey items, 81.3% to 100% of trainees responded “agree” or “strongly agree,” with the highest ratings observed for the usefulness of theoretical courses and the relevance of content to clinical work. Post-training assessments showed significant improvements in theoretical examination (median difference=24, 95% CI [23.3, 26.6], W=13,366), abnormal image interpretation (median difference=36, 95% CI [32.7, 36.7], W=13,041), and operational skills (median difference=30, 95% CI [28.7, 31.3], W=13,366) (all P <0.05). Median scores significantly improved post-training compared with baseline: clinical case assessment scores increased from 48 (IQR: 38, 55) to 76 (IQR: 72 to 84); case analysis from 11 (IQR: 8 to 15) to 21 (IQR: 17 to 26); and image interpretation from 4 (IQR: 2 to 7) to 16 (IQR 13 to 18) (all P <0.05). During clinical practice, a total of 956 critical care ultrasound examinations were performed. Of these, 415 examinations (43.41%) provided additional diagnostic clarification. Overall, 789 examinations (82.53%) generated ultrasound findings that supported, refined, or contextualized ongoing diagnostic assessment or clinical decision-making. Conclusions: Our longitudinal ultrasound training program strengthens foundational knowledge and enhances clinical application of critical care ultrasound. However, this single-center study lacks a comparator arm and long-term patient outcome evaluation. Future multicenter studies should validate these findings, compare training methodologies, and assess the impact of critical care ultrasound on patient morbidity and mortality.
INTRODUCTION:Grounded in Miller's pyramid, simulation seeks to provide a "shows how" learning experience, bridging theory to clinical application. This pilot study explores integrating role-play simulation-based learning within a third-year undergraduate speech-language pathology theory course at a South African university to address the theory-practice gap prevalent in traditional curriculum designs. METHODOLOGY:This study employed a multimethod approach, primarily using qualitative design and analytic approaches with a purposive sample. It incorporated role-play simulations focused on informal bedside cognitive-communication assessments for traumatic brain injury and motor speech disorders, complemented by an escape room activity simulating real-world case scenarios. Data were collected through pre- and postcourse questionnaires, self-reflections, and reflexive thematic analysis of simulation debriefs. RESULTS:Findings revealed that role-play simulations improved student knowledge, confidence, and integration of theoretical concepts, although some students expressed a need for additional practice and feedback. The structured design, peer and educator feedback during debriefings, and practical application were highlighted as critical factors contributing to learning. CONCLUSION:These results from this pilot study suggest that role-play simulations can be effectively implemented in theory courses, offering a valuable pedagogical tool to assist in closing the theory-practice gap.
The objective of this review is to identify and explore the key concepts and types of evidence for immersive virtual reality simulation design within the context of healthcare education. Immersive virtual reality simulation is being used increasingly within healthcare education, but there is a paucity of literature available to guide its effective design. A "Population, Context, and Concept" framework was established to identify articles about healthcare education and the design of immersive virtual reality simulation. In March 2023, a search was conducted using CINAHL, Medline, ERIC, and Scopus, where 13 articles were determined as eligible. This search was repeated in March 2024, where a further 15 articles were determined as eligible. The literature highlighted 3 themes, which included rationale, content design and modification, and simulation development. The emergent themes and their interactions form a framework that can be used by educators to guide the design of immersive virtual reality simulations.
