Introduction Effective communication is a core competency in medical practice, supporting patient relationships, treatment adherence, and clinical outcomes. Developing both skills and confidence is a central goal of medical education. Self-assessment, guided reflection, and exemplar-based feedback are promising strategies to enhance competence and self-regulation. Yet, evidence on medical students’ accuracy in self-evaluating communication remains limited. This study examined calibration of self-assessed communication performance, potential gender differences, impact of feedback strategies, and changes over time. Method We analyzed data from 78 students in a larger study who completed self-evaluation of their communication with a virtual patient. Participants engaged with eight video-based clinical vignettes targeting two communication competencies: providing clear, relevant information and responding to patient emotions. For each vignette, students recorded an initial response, reviewed exemplar feedback, completed a self-evaluation (with or without reflection prompts), and re-recorded their response. Communication behaviors were scored using an AI-based system validated against expert ratings. Mixed-effects regression models tested linear and quadratic effects of actual performance on self-assessed behaviors. Gender, experimental condition, and time (T1 vs. T2) were included as moderators to examine differences and changes in calibration. Results Students consistently overestimated their performance at low actual ability levels (intercept significantly greater than zero: t(154) = 13.48, p < .001, d = 1.53) and underestimated at higher levels (slope significantly less than one: t(148) = −9.99, p < .001, d = −1.13). The quadratic term did not improve model fit. No significant gender differences or improvements in calibration over time or feedback condition were observed. Conclusion Medical students show systematic miscalibration in self-assessed communication, even when engaging in structured self-evaluation and exemplar-based reflection. These findings highlight the importance of instructional strategies that actively promote comparison with high-quality exemplars and guided reflection. Integrating these approaches may reduce miscalibration, strengthen communication skills, and better prepare students for the interpersonal demands of clinical practice.

Medical error disclosure to patients is a critical skill that is often not taught effectively in medical training. The Video-based Communication Assessment (VCA) software enables trainees to receive feedback on their error disclosure communication skills. The VCA method also allows examination of the specific types of error disclosure responses that patients value most. The primary aim of this study was to describe the language medical residents use to disclose a hypothetical harmful medical error, and to determine the language associated with higher ratings by crowdsourced laypeople. A secondary aim of this study was to examine the alignment between error disclosure content recommended by experts and the communication behaviors that contribute to higher layperson ratings of disclosure. 102 resident physician responses to a case depicting a delayed diagnosis of breast cancer and their crowdsourced ratings were analyzed using thematic content analysis. We assessed the presence of specific themes in response to three sequential video prompts within a clinical case. Linear regressions were then performed for each prompt's response to examine the extent to which each theme predicted overall communication scores from layperson raters. Nearly all (N = 92, 90.2%) residents provided responses which included either a general apology or a specific apology in at least one of the three prompt's responses, and nearly all (N = 98, 96.1%) residents provided at least one response expressing a component of empathy. However, only 57.8% of residents openly acknowledged that the care was delayed, and 67.8% expressed a plan to prevent future errors. A few residents used rationalization (5.9%) or minimization (4.9%) behaviors; responses with these behaviors were associated with negative beta-coefficients, although this finding did not reach statistical significance. In a linear regression analysis, the strongest positive associations between resident responses and patient ratings were clustered around expressions of accountability (0.48), personal regret (0.47), apology (0.34), and intentions to prevent future mistakes (0.34). Resident physicians vary in which communication elements and themes they include during error disclosure, missing opportunities to meet patient expectations. While infrequent, some residents employed minimization or rationalization in their responses. Utilizing an assessment and feedback system that encourages responders to include themes layperson raters value most and to omit harmful expressions could be an important feature for future software for error disclosure communication training.

Objective Lung cancer screening (LCS) can detect lung cancer at an earlier stage and reduce lung cancer mortality but can also cause harm. Therefore, physicians are tasked with conveying complex information to help patients make informed decisions. Studies suggest physicians often fall short of this. We characterize how physicians respond to a patient reluctant to undergo LCS and identify communication elements patients respond favorably to. Method Physicians responded to a vignette of a reluctant patient. Analogue patients (APs) rated responses and described their preferred response. We analyzed both physician and AP qualitative responses using content analysis and compared mean ratings of physician responses with a particular communication element present to responses without that element. Results We identified 19 communication elements in 42 physician responses and 24 elements in 64 AP responses. We identified 5 elements that were associated with higher AP ratings and how often physicians engaged in them (e.g., showing desire to understand the patient as a person, occurring in 21% of responses.) Conclusion We identify communication elements in physician and AP responses and reveal gaps between patients' desires and strategies employed by physicians. Practical Implications Findings suggest that incorporating specific communication behaviors may enhance patient receptivity regarding LCS.

This paper presents an automated scoring approach for a formative assessment tool aimed at helping learner physicians enhance their communication skills through simulated patient interactions. The system evaluates transcribed learner responses by detecting key communicative behaviors, such as acknowledgment, empathy, and clarity. Built on an adapted version of the ACTA scoring framework, the model achieves a mean binary F1 score of 0.94 across 8 clinical scenarios. A central contribution of this work is the investigation of how to balance scoring accuracy with scalability. We demonstrate that synthetic training data offers a promising path toward reducing reliance on large, annotated datasets-making automated scoring more accurate and scalable.

Importance Residents must prepare for effective communication with patients after medical errors. The video-based communication assessment (VCA) is software that plays video of a patient scenario, asks the physician to record what they would say, engages crowdsourced laypeople to rate audio recordings of physician responses, and presents feedback to physicians. Objective To evaluate the effectiveness of VCA feedback in resident error disclosure skill training. Design, Setting, and Participants This single-blinded, randomized clinical trial was conducted from July 2022 to May 2023 at 7 US internal medicine and family medicine residencies (10 total sites). Participants were second-year residents attending required teaching conferences. Data analysis was performed from July to December 2023. Intervention Residents completed 2 VCA cases at time 1 and were randomized to the intervention, an individual feedback report provided in the VCA application after 2 weeks, or to control, in which feedback was not provided until after time 2. Residents completed 2 additional VCA cases after 4 weeks (time 2). Main Outcomes and Measures Panels of crowdsourced laypeople rated recordings of residents disclosing simulated medical errors to create scores on a 5-point scale. Reports included learning points derived from layperson comments. Mean time 2 ratings were compared to test the hypothesis that residents who had access to feedback on their time 1 performance would score higher at time 2 than those without feedback access. Residents were surveyed about demographic characteristics, disclosure experience, and feedback use. The intervention's effect was examined using analysis of covariance. Results A total of 146 residents (87 [60.0%] aged 25-29 years; 60 female [41.0%]) completed the time 1 VCA, and 103 (70.5%) completed the time 2 VCA (53 randomized to intervention and 50 randomized to control); of those, 28 (54.9%) reported reviewing their feedback. Analysis of covariance found a significant main effect of feedback between intervention and control groups at time 2 (mean [SD] score, 3.26 [0.45] vs 3.14 [0.39]; difference, 0.12; 95% CI, 0.08-0.48; P = .01). In post hoc comparisons restricted to residents without prior disclosure experience, intervention residents scored higher than those in the control group at time 2 (mean [SD] score, 3.33 [0.43] vs 3.09 [0.44]; difference, 0.24; 95% CI, 0.01-0.48; P = .007). Worse performance at time 1 was associated with increased likelihood of dropping out before time 2 (odds ratio, 2.89; 95% CI, 1.06-7.84; P = .04). Conclusions and Relevance In this randomized clinical trial, self-directed review of crowdsourced feedback was associated with higher ratings of internal medicine and family medicine residents' error disclosure skill, particularly for those without real-life error disclosure experience, suggesting that such feedback may be an effective way for residency programs to address their requirement to prepare trainees for communicating with patients after medical harm.

Background US residents require practice and feedback to meet Accreditation Council for Graduate Medical Education mandates and patient expectations for effective communication after harmful errors. Current instructional approaches rely heavily on lectures, rarely provide individualized feedback to residents about communication skills, and may not assure that residents acquire the skills desired by patients. The Video-based Communication Assessment (VCA) app is a novel tool for simulating communication scenarios for practice and obtaining crowdsourced assessments and feedback on physicians’ communication skills. We previously established that crowdsourced laypeople can reliably assess residents’ error disclosure skills with the VCA app. However, its efficacy for error disclosure training has not been tested. Objective We aimed to evaluate the efficacy of using VCA practice and feedback as a stand-alone intervention for the development of residents’ error disclosure skills. Methods We conducted a pre-post study in 2020 with pathology, obstetrics and gynecology, and internal medicine residents at an academic medical center in the United States. At baseline, residents each completed 2 specialty-specific VCA cases depicting medical errors. Audio responses were rated by at least 8 crowdsourced laypeople using 6 items on a 5-point scale. At 4 weeks, residents received numerical and written feedback derived from layperson ratings and then completed 2 additional cases. Residents were randomly assigned cases at baseline and after feedback assessments to avoid ordinal effects. Ratings were aggregated to create overall assessment scores for each resident at baseline and after feedback. Residents completed a survey of demographic characteristics. We used a 2×3 split-plot ANOVA to test the effects of time (pre-post) and specialty on communication ratings. Results In total, 48 residents completed 2 cases at time 1, received a feedback report at 4 weeks, and completed 2 more cases. The mean ratings of residents’ communication were higher at time 2 versus time 1 (3.75 vs 3.53; P<.001). Residents with prior error disclosure experience performed better at time 1 compared to those without such experience (ratings: mean 3.63 vs mean 3.46; P=.02). No differences in communication ratings based on specialty or years in training were detected. Residents’ communication was rated higher for angry cases versus sad cases (mean 3.69 vs mean 3.58; P=.01). Less than half of all residents (27/62, 44%) reported prior experience with disclosing medical harm to patients; experience differed significantly among specialties (P<.001) and was lowest for pathology (1/17, 6%). Conclusions Residents at all training levels can potentially improve error disclosure skills with VCA practice and feedback. Error disclosure curricula should prepare residents for responding to various patient affects. Simulated error disclosure may particularly benefit trainees in diagnostic specialties, such as pathology, with infrequent real-life error disclosure practice opportunities. Future research should examine the effectiveness, feasibility, and acceptability of VCA within a longitudinal error disclosure curriculum.

Background Residents may benefit from simulated practice with personalized feedback to prepare for high-stakes disclosure conversations with patients after harmful errors and to meet American Council on Graduate Medical Education mandates. Ideally, feedback would come from patients who have experienced communication after medical harm, but medical researchers and leaders have found it difficult to reach this community, which has made this approach impractical at scale. The Video-Based Communication Assessment app is designed to engage crowdsourced laypeople to rate physician communication skills but has not been evaluated for use with medical harm scenarios. Objective We aimed to compare the reliability of 2 assessment groups (crowdsourced laypeople and patient advocates) in rating physician error disclosure communication skills using the Video-Based Communication Assessment app. Methods Internal medicine residents used the Video-Based Communication Assessment app; the case, which consisted of 3 sequential vignettes, depicted a delayed diagnosis of breast cancer. Panels of patient advocates who have experienced harmful medical error, either personally or through a family member, and crowdsourced laypeople used a 5-point scale to rate the residents’ error disclosure communication skills (6 items) based on audiorecorded responses. Ratings were aggregated across items and vignettes to create a numerical communication score for each physician. We used analysis of variance, to compare stringency, and Pearson correlation between patient advocates and laypeople, to identify whether rank order would be preserved between groups. We used generalizability theory to examine the difference in assessment reliability between patient advocates and laypeople. Results Internal medicine residents (n=20) used the Video-Based Communication Assessment app. All patient advocates (n=8) and 42 of 59 crowdsourced laypeople who had been recruited provided complete, high-quality ratings. Patient advocates rated communication more stringently than crowdsourced laypeople (patient advocates: mean 3.19, SD 0.55; laypeople: mean 3.55, SD 0.40; P<.001), but patient advocates’ and crowdsourced laypeople’s ratings of physicians were highly correlated (r=0.82, P<.001). Reliability for 8 raters and 6 vignettes was acceptable (patient advocates: G coefficient 0.82; crowdsourced laypeople: G coefficient 0.65). Decision studies estimated that 12 crowdsourced layperson raters and 9 vignettes would yield an acceptable G coefficient of 0.75. Conclusions Crowdsourced laypeople may represent a sustainable source of reliable assessments of physician error disclosure skills. For a simulated case involving delayed diagnosis of breast cancer, laypeople correctly identified high and low performers. However, at least 12 raters and 9 vignettes are required to ensure adequate reliability and future studies are warranted. Crowdsourced laypeople rate less stringently than raters who have experienced harm. Future research should examine the value of the Video-Based Communication Assessment app for formative assessment, summative assessment, and just-in-time coaching of error disclosure communication skills.

Background Physician delivered weight management counseling (WMC) occurs infrequently and physicians report lack of training and poor self-efficacy. The purpose of this study was to develop and test the Video-based Communication Assessment (VCA) for weight management counseling (WMC) training in medical residents. Methods This study was a mixed methods pilot conducted in 3 phases. First, we created five vignettes based on our prior data and expert feedback, then administered the vignettes via the VCA to Internal Medicine categorical residents ( n = 16) from a University Medical School. Analog patients rated responses and also provided comments. We created individualized feedback reports which residents were able to view on the VCA. Lastly, we conducted debriefing interviews with the residents ( n = 11) to obtain their feedback on the vignettes and personalized feedback. Interviews were transcribed, and we used thematic analysis to generate and apply codes, followed by identifying themes. Results Descriptive statistics were calculated and learning points were created for the individualized feedback reports. In VCA debriefing interviews with residents, five themes emerged: 1) Overall the VCA was easy to use, helpful and more engaging than traditional learning and assessment modes, 2) Patient scenarios were similar to those encountered in the clinic, including diversity, health literacy and different stages of change, 3) The knowledge, skills, and reminders from the VCA can be transferred to practice, 4) Feedback reports were helpful, to the point and informative, including the exemplar response of how to best respond to the scenario, and 5) The VCA provide alternatives and practice scenarios to real-life patient situations when they aren’t always accessible. Conclusions We demonstrated the feasibility and acceptability of the VCA, a technology delivered platform, for delivering WMC to residents. The VCA exposed residents to diverse patient experiences and provided potential opportunities to tailor providers responses to sociological and cultural factors in WMC scenarios. Future work will examine the effect of the VCA on WMC in actual clinical practice.

Objective: Effective physician-patient communication is important, but physicians who are seeking to improve have few opportunities for practice or receive actionable feedback. The Video-based Communication Assessment (VCA) provides both. Using the VCA, physicians respond to communication dilemmas depicted in brief video vignettes; crowdsourced analog patients rate responses and offer comments. We characterized analog patients' comments and generated actionable recommendations for improving communication. Methods: Physicians and residents completed the VCA; analog patients rated responses and answered:"What would you want the provider to say in this situation?" We used qualitative analysis to identify themes. Results: Forty-three participants completed the VCA; 556 analog patients provided 1035 comments. We identified overarching themes (e.g., caring, empathy, respect) and generated actionable recommendations, incorporating analog patient quotes. Conclusion: While analog patients' comments could be provided directly to users, conducting a thematic analysis and developing recommendations for physician-patient communication reduced the burden on users, and allowed for focused feedback. Research is needed into physicians' reactions to the recommendations and the impact on communication. Practice implications: Physicians seeking to improve communication skills may benefit from practice and feedback. The VCA was designed to provide both, incorporating the patient voice on how best to communicate in clinical situations. (c) 2021 Published by Elsevier B.V.