Abstract Background: The importance of deliberate practice and feedback to promote transfer of learning is well established. Virtual Standardized Patients (VSPs) are an emerging modality designed to more realistically simulate patient encounter sections (versus traditional e-learning), while providing automatically generated feedback to the learner. This study aims to evaluate the effectiveness of VSPs on the performance of third year medical students in a clinical skills examination. Methods: The study was a post-test design with volunteer third-year medical students from a medical school in a large, urban, metropolitan area in Los Angeles, California. The entire class of medical students was given access to a virtual standardized patient from “Recourse AI” (AI-VSP) and their performance on subsequent clinical examinations (CSEs) was measured. A formative CSE was followed by a summative one three months later. Students were allowed to interact as many times with the AI-VSP as they desired, and received automatically generated feedback from the system. The AI-VSP presented with a chief complaint similar to that presented by a standardized patient (SP) in the clinical exams. We hypothesized that students who utilized the AI-VSP would perform better on that particular case, than those who did not. T-tests were used for statistical analysis. Results: Students who interacted with the AI-VSP before the formative CSE achieved significantly higher scores for the History section in the first clinical examination, when presented with a SP case with a similar chief complaint (p = 0.001). Those students who interacted with an AI-VSP more than once performed better than those who interacted only once. (p = 0.005). Scores on the summative CSE three months after the interaction showed that the history scores were still slightly higher for those who had interacted at least once with AI-VSP, although there was no longer a significant difference. Conclusions: The use of the AI-VSP showed a statistically significant improvement in history-taking skills as measured in a subsequent CSE. The improvement was not as significant in another exam three months later, underscoring the need for repetition and deliberate practice to maintain clinical skills, and maybe an estimation of the time interval required for repetition.

The discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills Examination emphasizes the need for other reliable standardized assessments of medical student clinical skills. For 30 years, the California Consortium for the Assessment of Clinical Competence (CCACC) has collaborated in the development of clinical skills assessments and has become a valuable resource for clinicians, standardized patient educators, psychometricians, and medical educators. There are many merits to strong multi-institutional partnerships, including the integration of data across multiple schools to provide feedback to both students and curricula, pooled test performance statistics for analysis and quality assurance, shared best practices and resources, individual professional development, and opportunities for research and scholarship. The infrastructure of the CCACC allows member schools to adapt to a changing medical landscape, from emerging trends in clinical medicine to the limitations imposed by a global pandemic. In the absence of a national examination, there is now a greater need for all medical schools to develop a comprehensive, dynamic, and psychometrically sound assessment that accurately evaluates clinical skills. Medical schools working together in regional consortia have the opportunity to create and implement innovative and robust assessments that evaluate a wide range of clinical skills, ensure that medical students have met an expected level of clinical competency before graduation, and provide a framework that contributes to ongoing efforts for the development of new national clinical skills standards.

Objective: In response to school shut downs amid the COVID-19 pandemic, nurse educators from the University of Southern California implemented a virtual objective structured clinical examination (OSCE) using standardized patients (SPs) to assess family nurse practitioner (FNP) students’ clinical and communication skills as an alternative to the traditional in-person OSCE format. The intent of this paper is to share the nurse educators’ experiences with the transitional process and students’ feedback about their virtual OSCE experiences.Methods: Students (N = 36) enrolled in a childbearing/childrearing clinical course participated in the virtual OSCE using Zoom. The experience included briefing and debriefing sessions. Students were evaluated for their communication and clinical decision making skills based on their assessment of two adolescent patients: one acute with behavioral problems presenting for a checkup and one with headache.Results: All students who participated in the virtual OSCE experience demonstrated appropriate clinical and communication skills. Students perceived the virtual OSCE as a realistic model for telehealth but missed social interaction with faculty and peers and found their inability to conduct physical exam maneuvers challenging. The majority (79.3%) preferred interacting with patients face-to-face.Conclusions: Virtual OSCEs used as low-stakes formative assessments provide FNP students with effective and valuable learning experiences. Transitioning from in-person to virtual OSCEs using Zoom is feasible but requires extensive collaboration between nursing educators and those with access to simulation facilities, such as faculty from schools of medicine. Findings from this experience will serve as a guide for deliberate process improvements for future iterations.

Physician biases toward mental conditions such as depression have been shown to adversely affect medical outcomes.

BackgroundImplicit attitudes are outside of conscious awareness and are thought to affect automatic responses outside of one's deliberate control, with the potential to impact physician-patient relationships.ObjectiveTo measure the nature and extent of implicit biases towards depression in internal medicine and psychiatry residents.DesignDescriptive and comparative study.ParticipantsFifty-one residents from three internal medicine programs and 35 residents from three psychiatry programs located in two states.InterventionsParticipants were sent a link to voluntarily participate in four online implicit association tests. Residents' identities were anonymous.Main MeasuresFour implicit association tests to measure the association of (1) attitude (good/bad), (2) permanence, (3) controllability, and (4) etiology with depression/physical illness.Key ResultsInternal medicine residents demonstrated a significant association between depression and negative attitudes (t(38)=6.01, p<.001, Cohen's d=.95), uncontrollability (t(35)=4.80, p<.001, Cohen's d=.79), temporariness (t(37)=2.94, p=.006, Cohen's d=.48), and a psychologic etiology (t(1)=6.91, p<.001, Cohen's d=1.24). Psychiatry residents only demonstrated an association between depression and a psychologic etiology (t(2)=4.79, p<.001, Cohen's d=4.5). When comparing the two specialties, internal medicine and psychiatry differed on two of the IATs. Internal medicine residents were more likely to associate negative attitudes with depression than psychiatry residents (t(63)=4.66, p<.001, Cohen's d=1.18) and to associate depression with being uncontrollable (t(57)=3.17, p=.002, Cohen's d=.81).ConclusionsInternal medicine residents demonstrated biases in their attitudes towards depression and significantly differed in some areas from psychiatry residents. This pilot study needs to be replicated to confirm our findings and further work needs to be done to determine the effect of these attitudes on the provision of clinical care.

Purpose To examine validity evidence of local graduation competency examination scores from seven medical schools using shared cases and to provide rater training protocols and guidelines for scoring patient notes (PNs). Method Between May and August 2016, clinical cases were developed, shared, and administered across seven medical schools (990 students participated). Raters were calibrated using training protocols, and guidelines were developed collaboratively across sites to standardize scoring. Data included scores from standardized patient encounters for history taking, physical examination, and PNs. Descriptive statistics were used to examine scores from the different assessment components. Generalizability studies (G-studies) using variance components were conducted to estimate reliability for composite scores. Results Validity evidence was collected for response process (rater perception), internal structure (variance components, reliability), relations to other variables (interassessment correlations), and consequences (composite score). Student performance varied by case and task. In the PNs, justification of differential diagnosis was the most discriminating task. G-studies showed that schools accounted for less than 1% of total variance; however, for the PNs, there were differences in scores for varying cases and tasks across schools, indicating a school effect. Composite score reliability was maximized when the PN was weighted between 30% and 40%. Raters preferred using case-specific scoring guidelines with clear point-scoring systems. Conclusions This multisite study presents validity evidence for PN scores based on scoring rubric and case-specific scoring guidelines that offer rigor and feedback for learners. Variability in PN scores across participating sites may signal different approaches to teaching clinical reasoning among medical schools.

The Research in Medical Education (RIME) Program Planning Committee is committed to advancing scholarship in and promoting dialogue about the critical issues of racism and bias in health professions education (HPE). From the call for studies focused on underrepresented learners and faculty in medicine to the invited 2016 RIME plenary address by Dr. Camara Jones, the committee strongly believes that dismantling racism is critical to the future of HPE. The evidence is glaring: Dramatic racial and ethnic health disparities persist in the United States, people of color remain deeply underrepresented in medical school and academic health systems as faculty, learner experiences across the medical education continuum are fraught with bias, and current approaches to teaching perpetuate stereotypes and insufficiently challenge structural inequities. To achieve racial justice in HPE, academic medicine must commit to leveraging positions of influence and contributing from these positions. In this Commentary, the authors consider three roles (educator, faculty developer, and researcher) represented by the community of scholars and pose potential research questions as well as suggestions for advancing educational research relevant to eliminating racism and bias in HPE.

This supplement includes the eight research papers accepted by the 2016 Research in Medical Education Program Planning Committee. In this Commentary, the authors use "conversations in medical education" as a guiding metaphor to explore what these papers contribute to the current scholarly discourse in medical education. They organize their discussion around two domains: the topic of study and the methodological approach. The authors map the eight research papers to six "hot topics" in medical education: (1) curriculum reform, (2) duty hours restriction, (3) learner well-being, (4) innovations in teaching and assessment, (5) self-regulated learning, and (6) learning environment, and to three purposes commonly served by medical education research: (1) description, (2) justification, and (3) clarification. They discuss the range of methods employed in the papers. The authors end by encouraging educators to engage in these ongoing scholarly conversations.

Background: Current scales for interprofessional team performance do not provide adequate behavioral anchors for performance evaluation. The Team Observed Structured Clinical Encounter (TOSCE) provides an opportunity to adapt and develop an existing scale for this purpose. We aimed to test the feasibility of using a retooled scale to rate performance in a standardized patient encounter and to assess faculty ability to accurately rate both individual students and teams.Methods: The 9-point McMaster-Ottawa Scale developed for a TOSCE was converted to a 3-point scale with behavioral anchors. Students from four professions were trained a priori to perform in teams of four at three different levels as individuals and teams. Blinded faculty raters were trained to use the scale to evaluate individual and team performances. G-theory was used to analyze ability of faculty to accurately rate individual students and teams using the retooled scale.Results: Sixteen faculty, in groups of four, rated four student teams, each participating in the same TOSCE station. Faculty expressed comfort rating up to four students in a team within a 35-min timeframe. Accuracy of faculty raters varied (38-81% individuals, 50-100% teams), with errors in the direction of over-rating individual, but not team performance. There was no consistent pattern of error for raters.Conclusion: The TOSCE can be administered as an evaluation method for interprofessional teams. However, faculty demonstrate a 'leniency error' in rating students, even with prior training using behavioral anchors. To improve consistency, we recommend two trained faculty raters per station.

In recent years, there has been a renewed interest in measuring perceptions regarding different aspects of the medical educational environment. A reliable tool was developed for measuring perceptions of the educational environment as it relates to evidence-based medicine as part of a multicountry randomised controlled trial to evaluate the effectiveness of a clinically integrated evidence-based medicine course. Participants from 10 specialties completed the questionnaire. A working dataset of 518 observations was available. Two independent subsets of data were created for conducting an exploratory factor analysis (n=244) and a confirmatory factor analysis (n=274), respectively. The exploratory factor analysis yielded five 67-item definitive instruments, with five to nine dimensions; all resulted in acceptable explanations of the total variance (range 56.6–65.9%). In the confirmatory factor analysis phase, all goodness-of-fit measures were acceptable for all models (root mean square error of approximation ≤0.047; comparative fit index≥0.980; normed χ² ≤1.647; Bentler-Bonett normed fit index ≥0.951). The authors selected the factorisation with seven dimensions (factor-7 instrument) as the most useful on pragmatic grounds and named it Evidence-Based Medicine Educational Environment Measure 67 (EBMEEM-67). Cronbach's α for subscales ranged between 0.81 and 0.93. The subscales are: ‘Knowledge and learning materials’; ‘Learner support’; ‘General relationships and support’; ‘Institutional focus on EBM’; ‘Education, training and supervision’; ‘EBM application opportunities’; and ‘Affirmation of EBM environment’. The EBMEEM-67 can be a useful diagnostic and benchmarking tool for evaluating the perceptions of residents of the environment in which evidence-based medicine education takes place.

Background: To understand how third-year medical student interprofessional collaborative practice (IPCP) is affected by self-efficacy and interprofessional experiences (extracurricular experiences and formal curricula).Methods: The authors measured learner IPCP using an objective structured clinical examination (OSCE) with a standardized nurse (SN) and standardized patient (SP) during a statewide clinical performance examination. At four California medical schools from April to August 2012, SPs and SNs rated learner IPCP (10 items, range 0-100) and patient-centered communication (10 items, range 0-100). Post-OSCE, students reported their interprofessional self-efficacy (16 items, 2 factors, range 1-10) and prior extracurricular interprofessional experiences (3 items). School representatives shared their interprofessional curricula during guided interviews.Results: Four hundred sixty-four of 530 eligible medical students (88%) participated. Mean IPCP performance was 79.6 +/- 14.1 and mean self-efficacy scores were 7.9 (interprofessional teamwork) and 7.1 (interprofessional feedback and evaluation). Seventy percent of students reported prior extracurricular interprofessional experiences; all schools offered formal interprofessional curricula. IPCP was associated with self-efficacy for interprofessional teamwork (beta = 1.6, 95% CI [0.1, 3.1], p = 0.04) and patient-centered communication (beta = 12.5, 95% CI [2.7, 22.3], p = 0.01).Conclusions: Medical student IPCP performance was associated with self-efficacy for interprofessional teamwork and patient-centered communication. Increasing interprofessional opportunities that influence medical students' self-efficacy may increase engagement in IPCP.

Abstract Introduction Self-assessment is a major component in medical education and professional development. Physicians can recognize their strengths and challenges through accurate self-assessment, enabling them to focus on needed areas for improvement. This exercise is designed to be conducted with third-year medical student approximately 12 weeks into the year, and is designed to address the ACGME competencies of patient care, interpersonal and communication skills, and practice-based learning and improvement. Methods This forty-five minute exercise incorporates a one-to-one student-faculty joint review of a video of a single student-patient encounter. Before the video is played, both the faculty member and student review the presenting situation depicted. The video is then viewed and both the student and faculty member and fill out provided feedback forms. The feedback is then shared and the student completes an action plan. Results In the evaluation of this exercise by the medical students, the quality of the session received an average score of 4.4 out of 5, and the amount of new information gained received a 4.2 out of 5. In examining the topics for the student action plans, approximately 45% of the items related to encounter management (primarily time management and organization of the encounter), 30% were related to the competency of patient care (divided between history, physical examination and desire to provide a better list of differential diagnoses); while the remaining 25% noted interpersonal or communication skills that could be optimized. Discussion The focus of this exercise is twofold: (1) student metacognitive skills and (2) basic doctor-patient encounter skills. The metacognitive skills include skills in self-review, self-assessment, and self-management. The doctor-patient encounter skills include the skills of eliciting a focused history, conducting a focused physical examination, and the usage of both case-specific interpersonal and communication skills.

Health care organizations face challenges in providing language services for Limited English proficiency (LEP) clients. Supported by a grant from the National Science Foundation, we have been working to develop a technology for proximate simultaneous medical interpretation. In an effort to understand the relative importance of physical proximity, audio cues and visual cues to effective interpretation, we conducted two controlled trials of the comparative impact on patient and provider satisfaction of four conditions which represent the interpretation circumstances with LEP patients and monolingual providers in hospital settings; a certified interpreter present in the consultation room (“In Person”); at a remote location mediated by audio only (“Telephone”); at a remote location mediated by audio and video (“Videoconference”), and no interpreter present (“No Interpreter). In study 1, dyads of a medical student and a standardized patient were randomly assigned to In Person or No Interpreter condition on a rotating basis, producing a total of 25 encounter sessions. In Study 2, four interpretation communication modes including Videoconference and Telephone condition simulated 25 encounters. Repeated measure one-way analyses of variance (ANOVA) showed preferences of patients and physicians for four different methods of interpretation. Patients expressed high satisfaction for their doctors regardless of the communication mode. Doctors’ perception of the interpretation quality was also as desirable in remote communication as onsite human interpretation. Patients reported significantly greater feelings of being guarded for their privacy and were more satisfied with the interpretation quality in the remote communication via telephone over in-person interpretation.

Empathy is one of the ‘‘hot’’ topics in medical education, as most educators believe it to be a humanistic attitude that students and physicians must have (Spiro 1992). It is considered to be an essential component of professional competence (Epstein et al. 2002). Indeed, empathy has been included as one of the attributes of ‘‘professionalism’’ by Canadian residents (Brownell & Cote 2001). Kohut states: ‘‘the empathic understanding of the experience of other human beings is as basic an endowment of man as his vision, hearing, touch, taste and smell’’ (Kohut 1971). Empathy can therefore be considered as one of the most important characteristics of a physician. Most research studies have used one of the following validated self-report instruments to measure the construct of empathy – the Jefferson Scale of Physician Empathy (JSPE) (Hojat et al. 2001), the Interpersonal Reactivity Index – Empathy Concern subscale (IRI-EC (Davis 1980), and the Balanced Emotional Empathy Scale (BEES) (Mehrabian 1996). Several studies that have examined selfperceived empathy of medical students using one of these three instruments, have reported that there is a decline in empathy during medical school, specifically as students move from pre-clinical to clinical years (Hojat et al. 2004; Austin et al. 2007; Chen et al. 2007; Newton et al. 2008; Stratton et al. 2008; Hojat et al. 2009). However, these findings have been challenged by other researchers, citing some methodological flaws (Colliver et al. 2010). Nonetheless, all agree on the importance of empathy in fostering the relationship between the patient and the doctor and the value of this relationship in providing better health care. The interest in developing interventions to increase medical students’ empathy can be traced back as early as 1989 (Kramer et al. 1989). In the current issue, two groups of researchers, one group from the United States and another from Australia reported their curricular interventions increased and maintained empathy in medical students, when measured with the JSE/JSPE-S. The article from Hojat and his colleagues focused on enhancing and sustaining empathy of medical students through the use of video-clips and movies. These authors designed a two-stage study where they first implemented a pre-post intervention to explore whether medical students’ self-perceived empathy could be increased by the intervention. They then implemented as second study ten weeks later, where students from the experimental group (from the first study) were randomly divided into two groups, with one of the groups receiving another intervention. The authors concluded that the interventions worked, as the group exposed to both these interventions had a higher score on the JSE than those students who were not. The authors acknowledged the need for a longer-term follow-up study as the interval between the two interventions was only ten weeks. The students in this study were pre-clinical medical students, and since most declines in empathy occurred during the clinical years, a follow-up of these students in the clinical years with a re-measure of empathy would be useful. Nevertheless, this is a valuable contribution to the literature, as the authors are asking us to think about how we can reinforce students’ empathy by the judicious placement of interventions in the curriculum by encouraging us to design empirical studies to determine where and when such interventions should occur. The second study by Hegazi and Wilson employs a crosssectional design to examine whether empathy varied in different years of medical school. The authors found that there was no significant difference in the empathy scores of the medical students in the different years of medical school. They also explored the relationship between empathy and some demographic variables including not only gender and age, (which have been examined in other studies), but also marital status, cultural beliefs, religious beliefs, and previous tertiary education. In order to promote and maintain empathy, programmes aimed at promoting personal and professional development were included in the curriculum as options for students. What I found interesting was that in their conclusions, the authors stated that empathy could be preserved in the medical school by careful student selection and the inclusion of personal and professional development programmes. The fact that they brought up student selection opens up new areas to explore. Although they did not mention the selection system that was used in their medical school, there has been a lot of discussion regarding the use of cognitive and non-cognitive characteristics of medical students for selection. It may be worthwhile to explore whether any of the assessment processes used in medical schools such as the Multiple Mini-Interview, have an association with the students’ self-reported empathy. The authors also acknowledged the limitations of a cross-sectional study, recommending that readers consider longitudinal studies.