There has been increased use of telemedicine in clinical practice, but telemedicine curricula are not standard in pediatric residency training. We developed a 2-case, Virtual Objective Structured Clinical Exam (VOSCE) for pediatric residents with telemedicine learning objectives. Resident competency was assessed using a behaviorally anchored checklist, and residents completed a pre/post self-assessment. Less than half of the residents met mastery for the following behaviors: confirmation of patient identifiers, optimization of technical aspects and partnering with the patient to perform a physical exam. Resident self-assessment improved after participating in the VOSCE but remained low in key domains such as virtual physical exam and mental health issue identification and guidance. Most pediatric residents have not mastered all the skills needed for a successful telemedicine encounter. Incorporating a curriculum focused on telemedicine best practices into residency training is highly merited.

Background and objective Governing boards require residents to undergo Resident-as-Teacher (RAT) training. Both Radiology residents (RRs) and program directors have reported the need for this training. However, existing literature on radiology curricula remains limited, and information on outcomes, innovative strategies, and feasibility is unclear. In light of this, we aimed to develop, implement, and evaluate a pilot RAT curriculum specific to radiology, based on Kern's curriculum model. Methods Based on the Kern model, a curriculum was developed and implemented in 2021-22 and evaluated using electronic pre- and post-module surveys assessing participating residents' self-reported knowledge, attitudes, and skills. Pre- and post-intervention results were reported as mean ± standard deviation (SD), and differences were analyzed using the Wilcoxon signed-rank test. Results Ten residents participated in the Teaching at the Workstation module, with 100% completing pre-and post-evaluation surveys. Residents felt more comfortable addressing gaps in medical students' (MS) skills (pre-test/post-test: 2.6 ± 0.52/3.1 ± 0.57, p<0.025), in understanding best-practice teaching (2.3 ± 0.48/3.1 ± 0.32, p<0.005) and more confident in teaching MS (2.6 ± 0.52/3.2 ± 0.42, p<0.034) and junior residents (2.6 ± 0.52/3.1 ± 0.32, p<0.025). Nine residents participated in the Delivering Effective Feedback module, with 56% completing pre- and post-evaluations. Residents' understanding of best-practice principles in delivering effective feedback improved, but not significantly (MS: 2.2 ± 0.45/3.3 ± 0.49, p=0.059. Nine residents participated in the Delivering a Case Presentation module, with 100% completing pre-and post-evaluations. They expressed increased comfort and confidence in developing (2.11 ± 0.33/2.91 ± 0.30, p=0.008), preparing (2.11 ± 0.60/2.91 ± 0.30, p=0.020), and delivering presentations (2.22 ± 0.66/3.82 ± 0.40, p=0.025). Conclusions A pilot radiology-specific RAT curriculum was successfully developed and implemented. Participating residents reported improvements across multiple knowledge, attitude, and skill domains. By using universal learning strategies and peer-reviewed content, the Kern-based curriculum is potentially generalizable to other specialties. Next steps include incorporating faculty development to support these curricula and assessing higher-level evaluation outcomes.

INTRODUCTION:Poor social connection is a risk factor for mortality comparable to smoking or obesity. Expert recommendations encourage clinicians to identify and address isolation, loneliness and absent social support; however, screening and documentation norms remain unstandardised in graduate medical education (GME) social determinants of health (SDOH) curricula. We assessed the knowledge, attitudes, and behaviours (KAB) of internal medicine resident physicians in a large, urban, academic medical centre regarding screening for and documenting social connection. METHODS:Between October 2022 and February 2023, a voluntary, anonymous mixed-methods survey was disseminated to assess resident physicians' KAB about social connection as a SDOH. Quantitative data were analysed descriptively. Qualitative responses underwent content analysis. An electronic medical record (EMR) chart review of resident notes corroborated survey data. RESULTS:Sixty-three residents of an eligible 153 responded to the survey (41% participation). Sixty-five per cent reported never receiving formal GME about social connection as a SDOH. Familiarity with validated screening tools was low. Although 51% of respondents reported often/always documenting patients' social support systems, few chart notes explicitly mentioned social contacts. Respondents agreed that physicians are responsible for assessing and documenting social connection and noted key challenges of time constraints, inadequate screening and documentation norms and EMR navigation difficulty. Targeted didactics, EMR improvements and interdisciplinary learning opportunities were identified as potential solutions. CONCLUSIONS:This study identified notable gaps in resident physician KAB to address social connection as a SDOH. Challenges could be addressed with standardised screening and documentation norms and skill development in patient communication and interdisciplinary teamwork.

ObjectivesCommunication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE.MethodsWe describe the implementation of the CCSAT including information on the underlying model, the tool’s items, domains, scales and scoring, and its role in curriculum. Internal structure is explored through item, internal consistency, and confirmatory factor analyses. Evidence for CCSAT validity is synthesized within prevailing frameworks (Messick12 and Kane13) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research.ResultsImplementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications.Conclusions and Practice ImplicationsThis pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.

Suboptimal support for colleagues experiencing discrimination can adversely impact clinician well-being and patient care. To describe resident performance and experience during an Objective Structured Clinical Examination (OSCE) case centered on supporting a trainee facing discrimination to inform enhanced, supportive learning environments. Formative, internal medicine OSCE at a simulation center. 148 second-year residents across 2018, 2019, 2021, 2022. Residents had 10 min to support a Muslim standardized intern (SI) experiencing discrimination from a patient. The SI rated resident performance across Supervision, Relationship Development, and Support domains and provided written feedback. Post-OSCE evaluations elicited resident reflections on case challenges. Proficient residents (≥ 80

Objective Unannounced standardized patients (USPs) have long been used to measure clinical performance in situ. These incognito actors capture data on clinician skills and patient experience. Given the widespread expansion of USPs, we conducted a systematic review with the goal of capturing USP uses across settings. Methods In collaboration with a medical librarian, we conducted searches across six databases. Data extracted included: target population, setting, and assessed skills. Reliability and fidelity measures, including detection, training methods, and assessment measures were also captured. Results 128 articles were included. Individual clinicians were the most frequently targeted (n=116, 91%). The collective care team was the target in one study (1%). Studies were primarily conducted in ambulatory settings (n=120, 93%). History gathering (n=91, 71%), communication (n=66, 52%), counseling (n=42, 33%), and patient education (n=46, 36%) were commonly assessed, as were correct diagnosis (n=33, 25.8%) and appropriate ordering of labs/tests (n=39, 31%).Detection information was not provided for 60 studies. Further, 22% of articles did not provide details on SP training. 82 (64%) reported case fidelity. Measures of inter-rater reliability were reported in 81 (63%) articles. Conclusions USPs capture a range of data domains but lack of uniform validity and reliability measures can undermine findings.

PurposeAs the COVID-19 pandemic forced most colleges and universities to go online, student health centers rapidly shifted to telehealth platforms without frameworks for virtual care provision. An urban student health center implemented a needs assessment involving unannounced standardized patients (USPs) to evaluate the integration of a new telehealth workflow and clinicians' virtual communication skills.MethodsFrom April to May 2021, USPs conducted two video visits with 12 primary care and four women's health clinicians (N = 16 clinicians; 32 visits). Cases included (1) a 21-year-old female presenting for birth control with a positive Patient Health Questionaire-9 and (2) a 21-year-old male, who vapes regularly, with questions regarding safe sex with men. Clinicians were evaluated using a checklist completed by the USP immediately following the visit and a systematic chart review of the electronic health record.ResultsUSP feedback indicates most clinicians received high ratings for general communication skills but may benefit from educational intervention in several key telemedicine skills. Clinicians struggled with using nonverbal signals to enrich communication (47% well done), acknowledging emotions (34% well done), and using video for information gathering (34% well done). Low rates of standard screenings (e.g., 63% administered the PHQ-2, <50% asked about alcohol use) suggested protocols for in-person care were not easily incorporated into telehealth practices, and clinicians may benefit from enhanced care team support. Performance reports were shared with clinicians and leadership postvisit.DiscussionResults suggest project design and implementation is scalable and feasible for use at other institutions, offering a structured methodology that can improve general student health care.

Background Residents lack confidence in caring for transgender individuals. More exposure and practice throughout training is needed. Objective To explore whether and how prior exposure to transgender health skills during medical school impacted competency with these skills during residency. Methods In 2022, all 101 internal medicine residents at New York University Grossman School of Medicine participated in an objective structured clinical examination (OSCE) station as part of their annual formative assessment where they cared for a standardized patient (SP) who identified as transgender. Three SPs who were members of the transgender community were recruited through online and social media forums. Two resident groups (continuum vs noncontinuum) differed in their prior experiences with transgender OSCEs during medical school. We analyzed SPs' ratings of resident performance using checklist data and SP open-ended feedback to compare performance between groups and resident post-OSCE evaluations to understand residents' perceptions of the educational value of the case. Results Residents with prior experience with transgender SPs (continuum) were more frequently recommended by SPs (88% [21 of 24] vs 70% [54 of 77]) to a family member or friend, were all rated professional (100% [24 of 24] vs 94% [72 of 94]) and scored better in pain information-gathering (92% vs 65%, mean summary score) and gender-affirming care skills (67% vs 52%, mean summary score). Noncontinuum residents lacked experience, missed opportunities to ask about gender identity, and needed work on demonstrating comfort and using proper language. Most residents completing a post-OSCE evaluation (80%, 41 of 51) rated the case as "very valuable." Conclusions Spaced practice and feedback through early exposure to transgender OSCEs were valuable for skill acquisition, giving continuum residents a learning advantage compared to noncontinuum residents.

Purpose Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents’ clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). Method EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children’s Hospital Medical Center, a quaternary children’s hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents’ rank after case-mix adjustments were also identified. Results 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0–1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2–21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations (r = 0.90–0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents’ absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. Conclusions For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents’ clinical performance and identify opportunities for quality improvement.

PURPOSE:To describe patterns of clinical communication skills that inform curriculum enhancement and guide coaching of medical students.MATERIALS AND METHODS:Performance data from 1182 consenting third year medical students in 9 cohorts (2011-2019), on a 17-item Clinical Communication Skills Assessment Tool (CCSAT) completed by trained Standardized Patients as part of an eight case high stakes Comprehensive Clinical Skills Exam (CCSE) were analyzed using latent profile analysis (LPA). Assessment domains included: information gathering (6 items), relationship development (5 items), patient education (3 items), and organization/time management (3 items). LPA clustered learners with similar strength/weakness into profiles based on item response patterns across cases. One-way analysis of variance (ANOVA) assessed for significant differences by profile for CCSAT items.RESULTS:Student performance clustered into six profiles in three groups, high performing (HP1 and HP2-Low Patient Education, 15.7%), average performing (AP1 and AP2-Interrupters, 40.9%), and lower performing profiles (LP1-Non-interrupters and LP2, 43.4%) with adequate model fit estimations and similar distribution in each cohort. We identified 3 CCSAT items that discriminated among learner's skill profiles.CONCLUSION:Clinical communication skill performance profiles provide nuanced, benchmarked guidance for curriculum improvement and tailoring of communication skills coaching.

The objective of this study was to compare unannounced standardized patient (USP) and patient reports of care. Patient satisfaction surveys and USP checklist results collected at an urban, public hospital were compared to identify items included in both surveys. Qualitative commentary was reviewed to better understand USP and patient satisfaction survey data. Analyses included χ 2 and Mann-Whitney U test. Patients provided significantly higher ratings on 10 of the 11 items when compared to USPs. USPs may provide a more objective perspective on a clinical encounter than a real patient, reinforcing the notion that real patients skew overly positive or negative.