INTRODUCTION: The national turnover rates for advanced practice providers (APPs) in 2016 was 12.6%, double that of physician turnover1. Lack of professional development and growth opportunities are among the reasons given for APPs to seek different employment opportunities.2 Utilizing 2020 Press-Ganey3 provider engagement scores in a single academic institution, it was found that 13 APPs within the Department of Anesthesiology had an engagement score of 4.57, on a scale of 1-5 (+0.56 > national average), and alignment score of 3.83 (+0.25 > national average). In an attempt to address the professional development needs of the APPs, an advancement pathway was developed to both recognize and reward these professionals. METHODS: The Division of Critical Care Medicine (CCM), within the Department of Anesthesiology, worked with the APPs to establish guidelines and criteria for the advancement pathway. This process helped to create buy-in from the team members and to choose meaningful criteria. Participation in the program was voluntary, consisting of three possible tiers with a requirement of at least 3-5 years of departmental service for eligibility. Criteria for each level tier included scholarly activity (peer-reviewed publications and research), educational progression, leadership involvement, clinical practice, and volunteerism. The department awarded participants for each given tier with additional professional days and a monthly stipend. The advancement pathway was introduced in July 2020, with guidelines for intent to apply to be submitted by December 2020, and an award date of July 2021. RESULTS: Four of eight eligible APPs applied for the advancement pathway. The 2021 Press-Ganey scores for CCM APPs, while showing a decrease in engagement, likely due to COVID, showed an increase in alignment. The engagement score was 4.42 (+0.52, compared to the organization), and the alignment score increased to 4.08 (+0.59, compared to the organization and +0.25 compared to 2020). CONCLUSION: Even through COVID, engagement scores remained high and alignment scores increased. Creation of the advancement pathway may enhance institutional alignment and potentially fosters departmental retention for APPs.

BACKGROUND High-quality and high-utility feedback allows for the development of improvement plans for trainees. The current manual assessment of the quality of this feedback is time consuming and subjective. We propose the use of machine learning to rapidly distinguish the quality of attending feedback on resident performance. METHODS Using a preexisting databank of 1925 manually reviewed feedback comments from 4 anesthesiology residency programs, we trained machine learning models to predict whether comments contained 6 predefined feedback traits (actionable, behavior focused, detailed, negative feedback, professionalism/communication, and specific) and predict the utility score of the comment on a scale of 1-5. Comments with ≥4 feedback traits were classified as high-quality and comments with ≥4 utility scores were classified as high-utility; otherwise comments were considered low-quality or low-utility, respectively. We used RapidMiner Studio (RapidMiner, Inc, Boston, MA), a data science platform, to train, validate, and score performance of models. RESULTS Models for predicting the presence of feedback traits had accuracies of 74.4%-82.2%. Predictions on utility category were 82.1% accurate, with 89.2% sensitivity, and 89.8% class precision for low-utility predictions. Predictions on quality category were 78.5% accurate, with 86.1% sensitivity, and 85.0% class precision for low-quality predictions. Fifteen to 20 hours were spent by a research assistant with no prior experience in machine learning to become familiar with software, create models, and review performance on predictions made. The program read data, applied models, and generated predictions within minutes. In contrast, a recent manual feedback scoring effort by an author took 15 hours to manually collate and score 200 comments during the course of 2 weeks. CONCLUSIONS Harnessing the potential of machine learning allows for rapid assessment of attending feedback on resident performance. Using predictive models to rapidly screen for low-quality and low-utility feedback can aid programs in improving feedback provision, both globally and by individual faculty.

Abstract Anesthesiologists are both teachers and learners and alternate between these roles throughout their careers. However, few anesthesiologists have formal training in the methodologies and theories of education. Many anesthesiology educators often teach as they were taught and may not be taking advantage of current evidence in education to guide and optimize the way they teach and learn. This review describes the most up-to-date evidence in education for teaching knowledge, procedural skills, and professionalism. Methods such as active learning, spaced learning, interleaving, retrieval practice, e-learning, experiential learning, and the use of cognitive aids will be described. We made an effort to illustrate the best available evidence supporting educational practices while recognizing the inherent challenges in medical education research. Similar to implementing evidence in clinical practice in an attempt to improve patient outcomes, implementing an evidence-based approach to anesthesiology education may improve learning outcomes.

BACKGROUND:Despite the critical nature of the residency interview process, few metrics have been shown to adequately predict applicant success in matching to a given program. While evaluating and ranking potential candidates, bias can occur when applicants make commitment statements to a program. Survey data show that pressure to demonstrate commitment leads applicants to express commitment to multiple institutions including telling >1 program that they will rank them #1. The primary purpose of this cross-sectional observational study is to evaluate the frequency of commitment statements from applicants to 5 anesthesiology departments during a single interview season, report how often each statement is associated with a successful match, and identify how frequently candidates incorrectly represented commitments to rank a program #1.METHODS:During the 2014 interview season, 5 participating anesthesiology programs collected written and verbal communications from applicants. Three residency program directors independently reviewed the statements to classify them into 1 of 3 categories; guaranteed commitment, high rank commitment, or strong interest. Each institution provided a deidentified rank list with associated commitment statements, biographical data, whether candidates were ranked-to-match, and if they successfully matched.RESULTS:Program directors consistently differentiated among strong interest, high rank, and guaranteed commitment statements with κ coefficients of 0.9 (95% CI, 0.8-0.9) or greater between any pair of reviewers. Overall, 35.8% of applicants (226/632) provided a statement demonstrating at least strong interest and 5.4% (34/632) gave guaranteed commitment statements. Guaranteed commitment statements resulted in a 95.7% match rate to that program in comparison to statements of high rank (25.6%), strong interest (14.6%), and those who provided no statement (5.9%). For those providing guaranteed commitment statements, it can be assumed that the 1 candidate (4.3%) who did not match incorrectly represented himself. Variables such as couples match, "R" positions, and not being ranked-to-match on both advanced and categorical rank lists were eliminated because they can result in a nonmatch despite truthfully ranking a program #1.CONCLUSIONS:Each level of commitment statement resulted in a progressively increased frequency of a successful match to the recipient program. Only 5.4% of applicants committed to rank a program #1, but these statements were very reliable. These data can help program directors interpret commitment statements and assist accurate evaluation of the interest of candidates throughout the match process.

Concussion is a common form of mild traumatic brain injury that can cause somatic, cognitive, and behavioral impairments lasting days to weeks. There are no published guidelines or recommendations to facilitate the safe and successful reintegration of anesthesiologist clinicians and trainees into clinical and academic work after concussion. We developed a simple 4-phase postconcussion recovery protocol for anesthesiologists who have suffered concussion and describe the successful use of this postconcussion recovery protocol to support reintegration of an Anesthesiology Critical Care Medicine fellow who developed mild concussion during vacation leave.

Anesthesiologists are both teachers and learners and alternate between these roles throughout their careers. However, few anesthesiologists have formal training in the methodologies and theories of education. Many anesthesiology educators often teach as they were taught and may not be taking advantage of current evidence in education to guide and optimize the way they teach and learn. This review describes the most up-to-date evidence in education for teaching knowledge, procedural skills, and professionalism. Methods such as active learning, spaced learning, interleaving, retrieval practice, e-learning, experiential learning, and the use of cognitive aids will be described. We made an effort to illustrate the best available evidence supporting educational practices while recognizing the inherent challenges in medical education research. Similar to implementing evidence in clinical practice in an attempt to improve patient outcomes, implementing an evidence-based approach to anesthesiology education may improve learning outcomes.

BACKGROUND: With the integration of Objective Structured Clinical Examinations into the Anesthesiology primary board certification process, residency programs may choose to implement Objective Structured Clinical Examinations for resident skill assessment. The aim of this study was to evaluate Objective Structured Clinical Examination–based milestone assessment and compare with Clinical Competency Committee milestone assessment that is based purely on clinical evaluations. METHODS: An annual Objective Structured Clinical Examination event was used to obtain milestone assessment of clinical anesthesia year 0–clinical anesthesia year 3 residents for selected milestones in patient care, professionalism, and interpersonal/communication skills. The Objective Structured Clinical Examination scenarios were different for each training level. The Clinical Competency Committee evaluated each resident semiannually based on clinical evaluations of resident performance. The Clinical Competency Committee milestone assessments from 2014 to 2016 that were recorded closest to the Objective Structured Clinical Examination event (±3 months) were compared to the Objective Structured Clinical Examination milestone assessments. A total of 35 residents were included in this analysis in 3 different training cohorts: A (graduates 2016, n = 12); B (graduates 2017, n = 10); and C (graduates 2018, n = 13). All residents participated in Objective Structured Clinical Examinations because their clinical anesthesia year 0 year and Clinical Competency Committee milestone data had been reported since December 2014. RESULTS: Both assessment techniques indicated a competency growth proportional to the length in training. Despite limited cumulative statistics in this study, average trends in the Objective Structured Clinical Examination–Clinical Competency Committee relationship indicated: (1) a good proportionality in reflecting competency growth; (2) a grade enhancement associated with Clinical Competency Committee assessment, dominated by evaluations of junior residents (clinical anesthesia year 0–clinical anesthesia year 1); and (3) an expectation bias in Clinical Competency Committee assessment, dominated by evaluation of senior residents (clinical anesthesia year 2–clinical anesthesia year 3). CONCLUSIONS: Our analysis confirms the compatibility of the 2 evaluation methods in reflecting longitudinal growth. The deviation of Objective Structured Clinical Examination assessments versus Clinical Competency Committee assessments suggests that Objective Structured Clinical Examinations may be providing additional or different information on resident performance. Educators might consider using both assessment methods to provide the most reliable and valid competency assessments during residency.

BACKGROUND:Flipped classroom (FC) is an active learning (AL) technique thought to have potential benefits in anesthesiology resident education. This survey aimed to determine the frequency of FC utilization, barriers of utilization, and means to overcome these barriers.METHOD:A web-based questionnaire was developed to survey anesthesiology faculty on their knowledge of and experience with FC. The Society of Academic Associations of Anesthesiology and Perioperative Medicine sent the survey to all United States core program directors (PD) via their list serve with a request for the PDs to forward the survey invitation to their clinical faculty. Descriptive statistics were summarized.RESULTS:A total of 244 anesthesiology faculty completed the survey. Reported faculty understanding of AL and FC were 57%. Of these faculty, 87% utilized AL and 57% utilized FC in their personal teaching practice during the past year (spring 2015-spring 2016). The most prevalent barriers to utilization of FC were faculty concern that learners would not come to class prepared or participate in class, faculty comfort with delivering traditional lectures, lack of faculty knowledge of necessary technological tools, and faculty concern about perceived increase in time needed to create a FC session. Eighty-nine percent of all faculty desired education on FC with preference for institutional workshops or grand rounds.CONCLUSIONS:Our survey found a discrepancy between faculty knowledge of FC and usage of this method in anesthesiology resident teaching. More educational resources are warranted to address barriers and familiarize faculty with FC applications in anesthesiology resident education.

Many educators are familiar with Objective Structured Clinical Examinations (OSCEs) as an element of skill assessment during medical school1 but the OSCE is not a common component of training or assessment for residents during graduate medical education in anesthesiology in the United States.2 The addition of OSCEs to the board certification process in anesthesiology is not a new concept. However, the OSCE format of the Applied Examination by the American Board of Anesthesiology (ABA) differs from previously implemented examination practices. The Royal College of Anaesthetists in the United Kingdom has a complex OSCE-based certification process (18 stations, 5 minutes each), covering a broad spectrum of skills including resuscitation, technical skills, anatomy, history taking, physical examination, communication skills, anesthetic hazards, the interpretation of x-rays, and utilizing high fidelity simulation.3 The Israeli Board of Anesthesiology certification includes a 5-station OSCE portion (15 minutes each), covering trauma management, resuscitation, crisis resource management in the operating room, regional anesthesia, and mechanical ventilation.4 In the United States, trainees and educators may be more familiar with the US Medical Licensing Examination (USMLE) OSCE process. The ABA OSCE differs from the USMLE OSCE by examination duration and complexity. While the USMLE OSCE is a much longer examination (approximately 8 vs 1.5 hours) and focuses on basic physician skills (focused history taking, physical examination skills, and telephone consultation encounter), the examination content of the ABA OSCE includes demonstration of technical anesthesia skills and an emphasis on higher level behaviors (communications and professionalism).5Figure 1.: The diagram describes the potential Objective Structured Clinical Examination (OSCE) benefits for the learner, the teacher, and the program.Although the recent inclusion of OSCEs in the Applied Examination by the ABA is likely to result in more programs providing their graduating residents with board preparation sessions for this new type of high-stakes examination, a recent survey demonstrated that only approximately 30% of anesthesiology residency training programs are currently using the OSCE as a part of their training curriculum.2 It is probable that many programs implementing OSCEs will utilize the ABA content outline5 and format to develop and provide practice experiences with the primary intention of preparing their graduating residents for the Applied Examination. We posit that there are other potential benefits of implementing OSCEs throughout the entire residency continuum beyond that of preparation for the ABA Applied Examination (Figure 1). While currently unrealized even to many educators, the benefits of OSCE implementation offer increased value to learners, educators, and training programs. THE “E” IN OSCE The current model of development and assessment of medical competence has been classified into 4 stages of learner capability: “knows,” “knows how,” “shows how,” and “does.”6 The most basic competency level, “knows,” is usually assessed by a knowledge test such as a multiple-choice examination. A higher level of medical competency, “knows how,” can be partially addressed in an oral examination setting.6,7 Several medical specialties have provided evidence that OSCEs can be used to test the higher levels of competency and the ABA has added an OSCE to the primary certification process in an attempt to assess the applicant’s competency at a level (“shows how”) not previously achieved.7 The ABA OSCE examination format includes professionalism and communication components, including informed consent, treatment options, periprocedural complications, ethics, communication with other professionals, and practice-based learning and improvement. It also includes technical skills and interpretation, such as the identification of anatomy via ultrasound and interpretation of transesophageal echocardiogram and patient monitors.5 After examining the reasons for implementing the OSCE and the competency assessment gaps that prompted the change in the ABA certification process, one might query if OSCEs are only beneficial as high-stakes summative examinations. We propose that OSCE-based assessment at recurring intervals throughout residency training could provide valuable information for the resident, teacher, and program in the progression toward competency. Individual residents may benefit from increased educational experiences.6 Educators may refine the experience of their interventions and their own teaching skills. Programs may learn more about best practices for training the next generation of anesthesiologists. BEYOND THE “E” IN OSCE The potential advantages of regular OSCE-based resident skill assessment during residency training go well beyond “E,” that is, the high-stakes summative examination. As a robust form of experiential learning that is objective, standardized, and clinically relevant, OSCEs used for formative assessment during training have benefits for the learner, the teacher, and program (Figure 1). For the Learner Skill Assessment. Summative evaluations may be used to measure the outcome of instruction at the end of a training period. A pass/fail format of OSCEs creates a high-stakes environment for the participant. However, in addition to summative evaluation, OSCE assessment can provide important formative feedback. The difference between summative and formative assessment is based on the desired use of the evaluation.8,9 By removing a simple pass/fail format, learners can receive specific feedback based on their performance. Such formative assessment would enrich the learning process for the learners and facilitate higher levels of performance.10Figure 2.: The OSCE definition can be expanded to address more learning opportunities. OSCE indicates Objective Structured Clinical Examination.By using OSCEs in a recurring fashion during training, learners receive frequent objective assessments that delineate skill levels and reveal deficits. For example, an OSCE requiring ultrasound identification of sciatic nerve anatomy on a simulated patient provides the learner an opportunity to self-reflect on this objective assessment of demonstrated skills apart from a pass/fail environment. With mentor assistance, the resident could develop individualized learning plans (ILPs) to fill any identified skill gaps. By expanding the number and frequency of OSCEs, learners can benefit from additional objective, standardized, and clinically relevant assessments and learning opportunities (Figure 2). Demonstration of Milestone Progression. Intermittent OSCE-based skill assessment may provide the learner with clear evidence of skill growth throughout the training process. By anchoring OSCE content within the specific Milestones (https://www.acgme.org/Portals/0/PDFs/Milestones/AnesthesiologyMilestones.pdf), learners may better perceive their own skill acquisition. Furthermore, they may be able to draw their own courses of study based on their individual performances. One example might be airway management and progression from recognition of adequacy of ventilation (Milestone level 1) to identifying and correcting problems associated with 1-lung ventilation (Milestone level 4). The objective, standardized, and clinically relevant characteristic of OSCE-based skill assessment may be advantageous for the learner and education process by relying on actual demonstration of proficiency to achieve specific Milestone levels. One obvious advantage for using simulation and formative OSCEs is the patient protection from ineptitude of a trainee learning new skills and encountering challenges beyond their current level of competence. Using repetitive OSCE-based assessment during the residency may provide more objective data for competency documentation, and may result in patients receiving more standardized care. Improved Learning and Retention. The learning gains of repeated testing versus repeated study (“testing effect”) have been demonstrated for knowledge acquisition and knowledge retention. Testing dramatically improves long-term memory for tested material.11 Repeated testing may also provide more favorable skills retention than repeated practice.12 Using the previous sciatic nerve block example, OSCEs can provide a blend of knowledge demonstration (identifying structures) and skills demonstration (needle positioning by ultrasound). Pairing a formative OSCE experience in a test–retest manner with purposeful and systematic practice in-between testing is an example of deliberate practice, a powerful way to improve performance. Therefore, an OSCE ought to be considered an experience that facilitates learning and not solely a form of assessment. Adding OSCE-based assessment to a multimodal teaching and learning environment (including clinical practice, didactics, and simulation) can facilitate the learning process at multiple stages of development for both skills and knowledge.13 Feedback, Self-Reflection, and Goal Setting. There are challenges to providing learners with sufficient feedback during clinical training.9 OSCEs offer the opportunity to provide feedback to the learner in a formative manner after direct observation of skill performance and with the goal of narrowing the gap between actual and desired performance. Learners may benefit from feedback that inherently compares them to others who have undergone the same OSCE.10 Furthermore, a primary source of learning is the result of reflective practices.14 OSCEs allow an environment for this to occur. The availability of objective data and comparison of individual performance with peers at similar level of training can help determine program- and specialty-specific performance expectations.15 However, embedding and encouraging self-reflection in the learning environment will promote life-long learning, and planned times for developing this skill before and after OSCEs might provide significant benefit for trainees.16 More specifically, OSCEs can allow the use of ILPs based on each learner.17 With a specific study plan for each resident, benefits for education can be maximized with targeted educational activities. An ILP using the mnemonic “Important, Specific, Measurable, Accountability, Realistic, Timeline” has been described as a successful approach.18 For example, instead of offering a central venous catheter workshop where all residents receive identical didactic and skills training, OSCE performance creates the chance for individualized educational intervention for catheter placement; 1 resident may need more help with ultrasound skills while another may need more time with modified Seldinger technique and suturing. Areas of needed remediation can be discovered. An OSCE assessment can provide measurable data for the ILP, generate realistic goals, and create an environment of accountability in a defined time frame. Over the course of a residency, trainee progression through Milestones can both be demonstrated and facilitated. OSCE Practice. One expected benefit of using OSCEs during training is the opportunity for deliberate practice with formative assessment before high-stakes summative assessment. A recent survey documented that many anesthesiology residency programs are planning to offer OSCEs for their senior residents with intent to prepare for primary board certification.2 However, OSCEs such as those used by the ABA require complex evaluation of validity and reliability in both the creation of the testing environment and in the assessment tools used.19 Such a detailed format may not be required to serve as practice opportunity. OSCEs used for formative assessment do not require the same psychometric evaluation. As such, many programs could provide OSCEs that are high-quality, that are appropriate for formative assessment, and that allow for deliberate practice without needing to have them undergo testing at the same level as that required for summative assessments. For the Educator Assessing Competence. Educators can use OSCE performance data to assess learner competence and identify skill gaps. This stems naturally from the unique value OSCEs offer residents. The difference here is that in addition to assisting an individual resident based on OSCE performance progression, program directors and educators can collect population data allowing for best education practices of groups of residents. That is, the performance of entire groups of residents on OSCEs (eg, by training level or entire program) can serve as a feedback mechanism for educators on how effectively they are teaching certain aspects of knowledge, skill, and practice throughout residency and whether certain pedagogical approaches are more effective than others. Direct clinical supervision and faculty intervention may mask deficits in learner clinical skills. The random nature of case loads and call duties may allow inequalities in experience that translate into inequalities in education. Furthermore, personality differences between faculty and residents have been shown to affect subjective evaluations.20 An OSCE curriculum for a residency program would allow assessment and demonstration of necessary skills. For example, an OSCE might require an advanced resident to evaluate a simulated patient with a periprocedural complication, conduct a focused evaluation, formulate an action plan, and discuss their plan with the patient. Depending on their performance, residents might demonstrate that their prior training is sufficient for this task. If all CA-3s perform at a Milestone level of 3 or 4, while most CA-1s perform at a level of 2, a program director could consider how faculty are educating residents in this skill; however, the program director might also be satisfied that the global experiences between these 2 years remains sufficient for achieving appropriate progression over time for this milestone. In fact, as skills are expected to grow over time, tailoring such OSCE assessments to expected Milestone progression would be an objective improvement in learner and educator assessment in many programs. Protecting Patients. OSCEs allow the teacher to adjust the level of supervision relative to the level of training and expected skill without having the ethical dilemma of needing to avoid patient harm. Thus, the high-stakes nature of some anesthetic situations can be taught and tested in a risk-free environment. Furthermore, these experiences can be repeated over and over without any risk to a patient and thus without the ethical dilemma of doing this in the clinical setting where such “practice” brings no benefit to the patient.21 Improving Teaching. When an educator administers and evaluates an OSCE over time, he or she will also be learning. Although the focus at first seems primarily on the learner, an educator benefits through the evaluation of teaching efficacy and through mastery of the given topic.22 Based on this feedback, the teacher can modify not only the OSCE itself but also his or her depth of knowledge and efficacy of interaction with the learners. For the Program Longitudinal Skill Assessment and Milestones. The Anesthesiology Milestones Project defined 25 subcompetencies with corresponding Milestones grouped into the 6 general competencies. When OSCE experiences are designed with the Milestones as a guide, learner performance data from OSCE assessment can help the Clinical Competency Committee assign appropriate Milestone levels that are reported to the ACGME, then shared with the ABA. Due to the ACGME requirement to report Milestone performance levels biannually for each learner, programs may find it difficult to obtain sufficient evaluative data for every subcompetency. OSCE-based assessment can supplement clinical evaluations and provide an opportunity to address accreditation required experiences that may be difficult for a program to offer as predictable encounters in the clinical environment for all residents. Furthermore, due to the standardized nature of the OSCE format, repetitive assessment may be a more trustworthy approach for objectively measuring growth in knowledge, skills, and practice. For example, it may be difficult to allow residents to demonstrate the ability to disclose an adverse event (Milestone Interpersonal and Communication Skills 1) or to manage ethical dilemmas (Milestone Professionalism 2) with independence (level 4). When considering the fifth subcompetency of the “patient care” general competency, program directors may not have observable data about level 4 performance (“identifies and manages clinical crises appropriately with conditional independence; assumes increasing responsibility for leadership of crisis response team”). An OSCE with a very scripted crisis would allow a resident to show graded progression from level 1 to level 4, the goal of the next accreditation system with the Milestones. By nature, these kind of events occur randomly in the clinical setting and often the resident will be influenced by the faculty who is present during such encounters. Additionally, programs may choose to design OSCEs to address program-specific Milestone gaps, therefore ensuring that the residents will have sufficient educational opportunities to develop and demonstrate competence in these areas. Curriculum Evaluation and Needs Assessment. An OSCE can be designed to evaluate a specific curriculum to determine if its educational goals were achieved.23,24 For example, the ABA OSCE Content Outline includes the application of ultrasonography for identifying relevant anatomy when performing transversus abdominis plane block. The collective performance of a class of residents may demonstrate adequacy of a regional anesthesia curriculum for this procedure. Conversely, if curriculum deficiencies are identified, interventions can be designed to address the deficit by adding clinical exposure or creating a training workshop. Using this aspect of OSCEs can aid in repetitive programmatic needs assessment to identify “gaps” between the current and desired conditions, a best practice when planning future educational offerings. In fact, this may encourage educators and programs to uncover and address “unperceived gaps” that formerly would have been overlooked. For example, an OSCE was recently used to evaluate point-of-care ultrasound skills of residents in 1 training program. The OSCE performance of the residents demonstrated a need to implement a comprehensive curriculum in point-of-care ultrasound skills.23 Subsequent measurements obtained by using the same OSCE over time may allow a program to document that learning gaps have been closed.24 FUTURE DIRECTIONS Based on a recent survey, 89% of anesthesiology residency program directors agreed that it is important to practice OSCEs for preparation for the ABA Applied Certification Examination. However, only 31% were currently providing mock OSCE experiences. Of those not providing mock OSCE experiences, 75% reported plans to start one.2 Most anesthesiology training programs are expected to move beyond the barriers of financial resources, faculty expertise, and time to implement OSCEs to prepare their residents for the ABA certification examination. Some of the barriers to OSCE development might be overcome by establishing a national working group formed to create and share OSCEs that include valid and reliable assessment tools. Development of scoring rubrics and implementation with consistency across raters remains a challenge for those who wish to move beyond using the Milestones as a key scoring guide. Further research and development should explore the specific content and subcompetencies best addressed by OSCEs. In addition, the question of how to appropriately integrate OSCEs as a teaching and assessment method needs to be answered. OSCEs may provide a satisfactory way to meet the elusive need of serial competency assessments throughout resident training. Moving beyond the mere examination understanding of “E” in OSCE could create multifaceted and significant educational benefits for the learners, teachers, programs, and profession. DISCLOSURES Name: Annette Rebel, MD Contribution: This author helped with the conception and design of the article, and to write the article. Name: Douglas L. Hester, MD. Contribution: This author helped with the conception and design of the article, and to write the article. Name: Amy DiLorenzo, MA. Contribution: This author helped with the conception and design of the article, and to write the article. Name: Matthew D. McEvoy, MD. Contribution: This author helped with the conception and design of the article, and to write the article. Name: Randall M. Schell, MD, MACM. Contribution: This author helped with the conception and design of the article, and to write the article. This manuscript was handled by: Edward C. Nemergut, MD.

Background Educational research projects are often developed and implemented at a single institution. However, the research project methods and results may not be generalizable and able to be replicated successfully at other institutions. The aim of this study was to investigate the process of replicating an effective educational Objective Structured Clinical Examination (OSCE) event at multiple other institutions. Methods An OSCE event was initially designed and implemented at the primary institution to assess the skill level of junior residents on the performance of basic anesthesia tasks. After the initial implementation, additional institutions were recruited to participate in a replication of this OSCE event at their own institutions. The primary institution provided the OSCE scenarios, assessment tools, rater training, and resident participant instructions. The participating secondary institutions' (n = 4) event managers obtained Institutional Review Board [IRB] approval, developed the event schedule, assigned faculty evaluators, and organized the simulation space at their own medical centers. The events were assessed by the secondary institutions' resident and faculty participants via an anonymous survey regarding the event's content and their perception of its educational value. Results We replicated a complex educational OSCE event, developed and implemented at 1 institution, at 4 other institutions. Resident participants (n = 60), participating faculty (n = 24), and event directors (n = 4) indicated a high level of appreciation for the OSCE event. Conclusion Using a structured approach, educational OSCE events can be successfully replicated at multiple institutions. Organization of multi-institutional studies and collaborative efforts is complex. This study illustrates 1 example of how to successfully approach multi-institutional educational projects.

BACKGROUND:In a flipped classroom approach, learners view educational content prior to class and engage in active learning during didactic sessions.OBJECTIVE:We hypothesized that a flipped classroom improves knowledge acquisition and retention for residents compared to traditional lecture, and that residents prefer this approach.METHODS:We completed 2 iterations of a study in 2014 and 2015. Institutions were assigned to either flipped classroom or traditional lecture for 4 weekly sessions. The flipped classroom consisted of reviewing a 15-minute video, followed by 45-minute in-class interactive sessions with audience response questions, think-pair-share questions, and case discussions. The traditional lecture approach consisted of a 55-minute lecture given by faculty with 5 minutes for questions. Residents completed 3 knowledge tests (pretest, posttest, and 4-month retention) and surveys of their perceptions of the didactic sessions. A linear mixed model was used to compare the effect of both formats on knowledge acquisition and retention.RESULTS:Of 182 eligible postgraduate year 2 anesthesiology residents, 155 (85%) participated in the entire intervention, and 142 (78%) completed all tests. The flipped classroom approach improved knowledge retention after 4 months (adjusted mean = 6%; P = .014; d = 0.56), and residents preferred the flipped classroom (pre = 46%; post = 82%; P < .001).CONCLUSIONS:The flipped classroom approach to didactic education resulted in a small improvement in knowledge retention and was preferred by anesthesiology residents.

BACKGROUND: Despite its importance, training faculty to provide feedback to residents remains challenging. We hypothesized that, overall, at 4 institutions, a faculty development program on providing feedback on professionalism and communication skills would lead to (1) an improvement in the quantity, quality, and utility of feedback and (2) an increase in feedback containing negative/constructive feedback and pertaining to professionalism/communication. As secondary analyses, we explored these outcomes at the individual institutions. METHODS: In this prospective cohort study (October 2013 to July 2014), we implemented a video-based educational program on feedback at 4 institutions. Feedback records from 3 months before to 3 months after the intervention were rated for quality (0–5), utility (0–5), and whether they had negative/constructive feedback and/or were related to professionalism/communication. Feedback records during the preintervention, intervention, and postintervention periods were compared using the Kruskal-Wallis and &khgr;2 tests. Data are reported as median (interquartile range) or proportion/percentage. RESULTS: A total of 1926 feedback records were rated. The institutions overall did not have a significant difference in feedback quantity (preintervention: 855/3046 [28.1%]; postintervention: 896/3327 [26.9%]; odds ratio: 1.06; 95% confidence interval, 0.95–1.18; P = .31), feedback quality (preintervention: 2 [1–4]; intervention: 2 [1–4]; postintervention: 2 [1–4]; P = .90), feedback utility (preintervention: 1 [1–3]; intervention: 2 [1–3]; postintervention: 1 [1–2]; P = .61), or percentage of feedback records containing negative/constructive feedback (preintervention: 27%; intervention: 32%; postintervention: 25%; P = .12) or related to professionalism/communication (preintervention: 23%; intervention: 33%; postintervention: 24%; P = .03). Institution 1 had a significant difference in feedback quality (preintervention: 2 [1–3]; intervention: 3 [2–4]; postintervention: 3 [2–4]; P = .001) and utility (preintervention: 1 [1–3]; intervention: 2 [1–3]; postintervention: 2 [1–4]; P = .008). Institution 3 had a significant difference in the percentage of feedback records containing negative/constructive feedback (preintervention: 16%; intervention: 28%; postintervention: 17%; P = .02). Institution 2 had a significant difference in the percentage of feedback records related to professionalism/communication (preintervention: 26%; intervention: 57%; postintervention: 31%; P < .001). CONCLUSIONS: We detected no overall changes but did detect different changes at each institution despite the identical intervention. The intervention may be more effective with new faculty and/or smaller discussion sessions. Future steps include refining the rating system, exploring ways to sustain changes, and investigating other factors contributing to feedback quality and utility.