Point-of-care ultrasound (POCUS) is an important clinical skill in internal medicine (IM) and requires robust methods to assess competency. The objectives of this research were to develop an entrustable professional activity (EPA) framework and instrument for POCUS competency assessment. An expert panel of seven IM POCUS educators guided the development of the EPA framework and assessment instrument. We evaluated the EPA framework and assessment instrument for validity, reliability, and feasibility of use for IM learners. Trained raters assessed POCUS competency in IM learners at the bedside. We performed a total of 604 assessments on 48 unique learners across three US academic sites. Authors analyzed the data using correlation, generalizability (G-) study, and decision (D-) study statistics to generate evidence for the validity, reliability, and feasibility of the instrument. The authors developed the EPA, “Assessing the acutely ill patient using POCUS,” using a structured process. The study team performed assessments on the ten clinical syndromes identified by the expert panel and single organ practice studies. The variance component attributed to learners ranged from 27.7 to 31.4

OBJECTIVES Lack of a well-functioning institutional feedback culture can undermine acquisition of skills essential for high quality patient care. The objective of this study was to assess feedback culture perceived by resident and fellow trainees, utilizing a mixed methods design. METHODS Pediatric fellows and residents completed an anonymous feedback environment survey consisting of 7 constructs: source credibility, feedback quality, feedback delivery, reinforcing feedback, constructive feedback, source availability, and promotion of feedback seeking, using a 7-point Likert scale. Trainee ratings were compared using two-sided Fisher’s exact tests. Multivariable analyses used a linear regression model. For the qualitative study, semistructured interviews of residents were conducted. The constant comparative method was used to incrementally code, categorize data, and derive themes. RESULTS Fifty-two residents and 21 fellows completed the survey (response rates 65% and 47%, respectively). Scores were more favorable for fellows compared with residents in 6 of 7 feedback constructs (P < .05), including on multivariate analysis. Hispanic ethnicity and female gender were associated with lower scores on source credibility (P = .04) and constructive feedback (P = .03), respectively. Two qualitative themes were identified: expectation of efficiency in patient care compromises the quality and quantity of feedback, and a culture that prioritizes courtesy over candor negatively impacts feedback quality. These themes were more pronounced when residents worked with pediatric subspecialists compared with hospitalists. CONCLUSIONS We described the feedback culture, which was less favorable in the residency program. The need for efficient patient care and a culture of courtesy adversely impacted the quality of feedback, especially among subspecialists.

Purpose In 2014, the Association of American Medical Colleges defined 13 Core Entrustable Professional Activities (EPAs) that all graduating students should be ready to do with indirect supervision upon entering residency and commissioned a 10-school, 5-year pilot to test implementing the Core EPAs framework. In 2019, pilot schools convened trained entrustment groups (TEGs) to review assessment data and render theoretical summative entrustment decisions for class of 2019 graduates. Results were examined to determine the extent to which entrustment decisions could be made and the nature of these decisions. Method For each EPA considered (4–13 per student), TEGs recorded an entrustment determination (ready, progressing but not yet ready, evidence against student progressing, could not make a decision); confidence in that determination (none, low, moderate, high); and the number of workplace-based assessments (WBAs) considered (0–>15) per determination. These individual student-level data were de-identified and merged into a multischool database; chi-square analysis tested the significance of associations between variables. Results The 2,415 EPA-specific determinations (for 349 students by 4 participating schools) resulted in a decision of ready (n = 997/2,415; 41.3%), progressing but not yet ready (n = 558/2,415; 23.1%), or evidence against student progression (n = 175/2,415; 7.2%). No decision could be made for the remaining 28.4% (685/2,415), generally for lack of data. Entrustment determinations’ distribution varied across EPAs (chi-square P < .001) and, for 10/13 EPAs, WBA availability was associated with making (vs not making) entrustment decisions (each chi-square P < .05). Conclusions TEGs were able to make many decisions about readiness for indirect supervision; yet less than half of determinations resulted in a decision of readiness to perform this EPA with indirect supervision. More work is needed at the 10 schools to enable authentic summative entrustment in the Core EPAs framework.

This Personal View is about our experience with preclinical education as medical students. We discuss the problem with current medical education in light of an ever-growing body of medical knowledge and increasing student disengagement with preclinical lectures. We briefly review the concept of retrieval practice as an effective, evidence-based learning strategy that helped us retain knowledge for longer periods and propose that medical educators should adopt this strategy to best prepare medical students to navigate the vastly expanding scope of modern medicine.

Background Reliable assessments of clinical skills are important for undergraduate medical education, trustworthy handoffs to graduate medical programs, and safe, effective patient care. Entrustable professional activities (EPAs) for entering residency have been developed; research is needed to assess reliability of such assessments in authentic clinical workspaces. Design A student-driven mobile assessment platform was developed and used for clinical supervisors to record ad hoc entrustment decisions using the modified Ottawa scale on 5 core EPAs in an 8-week internal medicine (IM) clerkship. After a 12-month period, generalizability (G) theory analysis was performed to estimate the reliability of entrustment scores and determine the proportion of variance attributable to the student and the other facets, including particular EPA, evaluator type (attending versus resident), or case complexity. Decision (D) theory analysis determined the expected reliability based on the number of hypothetical observations. A g-coefficient of 0.7 was used as a generally agreed upon minimum reliability threshold. Key Results A total of 1368 ratings over the 5 EPAs were completed on 94 students. Variance attributed to person (true variance) was high for all EPAs; EPA-5 had the lowest person variance (9.8% across cases and four blocks). Across cases, reliability ranged from 0.02 to 0.60. Applying this to the Decision study, the estimated number of observations needed to reach a reliability index of 0.7 ranged between 9 and 11 for all EPAs except EPA5 which was sensitive to case complexity. Conclusions Work place-based clinical skills in IM clerkship students were assessed and logged using a convenient mobile platform. Our analysis suggests that 9-11 observations are needed for these EPA workplace-based assessments (WBAs) to achieve a reliability index of 0.7. Note writing was very sensitive to case complexity. Further reliability analyses of core EPAs are needed before US medical schools consider wider adoption into summative entrustment processes and GME handoffs.

Welcome to the rewarding practice of HIV medicine. We need you and many more like you. Persons living with HIV (PLWH), in many parts of the USA, are having trouble finding doctors as a workforce shortage looms. With the success of antiretroviral therapy (ART) in decreasing mortality, the prevalence of HIV is rising. However, the ‘pipeline’ is not supplying enough physicians to meet this growing demand. Several factors account for the waning popularity of HIV medicine. The apprehension I frequently hear relates to the evolving clinical demands in caring for PLWH. The typical HIV patient in 2020 is no longer an otherwise healthy young person. On the contrary, he or she is ageing, infirmed with chronic diseases, vulnerable to polypharmacy and encumbered with psychosocial problems. And the practice environment is often beset with declining remuneration, increasing caseloads and voluminous clerical tasks. In short, in the mind of some physicians, the once exciting ‘clean’ practice of HIV medicine is now ‘muddied’ with the drudgery of ‘primary care’. HIV, it seems, has lost its sex appeal. Perhaps I am in the minority but, for me, caring for PLWH remains rewarding. So, let me share a perspective that counters this discouraging narrative and might sustain your youthful enthusiasm. First, resist the tendency to become inured to the marvels of ART. I see this often as clinicians monitor their patients’ responses. They ‘celebrate’ an undetectable viral load with no more excitement than they would upon restoring serum potassium in a patient taking a diuretic. Take a moment to consider this stunning achievement. A well tolerated, single, coformulated pill, taken daily, ‘miraculously’ transforms an inexorably fatal disease into a manageable chronic illness with a nearly normal life expectancy. Share in your patients’ joy in receiving this news. Revel in the exhilarating pace of progress in HIV medicine. Basic virology discoveries translate quickly into clinical trials and then into clinical practice. The protease inhibitor ‘epoch’, in which patients suffered from lipodystrophy, proved fleeting as integrase strand inhibitors supplanted protease inhibitors as preferred agents. With so many therapeutics emerging, the ink rarely dries on clinical guidelines. Embrace complexity. Before prescribing elvitegravir--cobicistat--tenofovir alafenamide--emtricitabine (in a single pill) to a 68-year-old man, you must contemplate multiple implications. Will the calcium he takes for osteoporosis interfere with the absorption of elvitegravir? Will the cobicistat inhibit the metabolism of the fluticasone he takes for his obstructive lung disease inhaler? To stay invigorated, you must awaken your inner internist and view these issues as intellectual challenges rather than burdensome complications. Care for the whole person. Patients become quite attached to their ‘HIV doctor’ and would rather not seek the remainder of their care elsewhere. Your patients will invite you into their lives, sharing their relationships, work-life, aspirations and regrets. This rich context strengthens your therapeutic alliance, enhances your shared decision-making and provides buoyancy during long hectic days. As I mentioned, the practice environment for HIV medicine is challenging. I recommend joining a practice that operates as a patient-centred medical home. The patient remains at the centre surrounded by concentric circles of support. The care team members practice at the upper limit of their training but remain willing to cheerfully pitch in to meet any of the patients’ needs. The Ryan White Program generously funds these type of services. Just prepare yourself for regular reporting requirements. Continue your civic engagement. As someone who has attended a few rallies, written a few editorials and joined a few campaigns, you will find kindred spirits in the HIV community. Epidemics expose underlying structural racism and marginalized populations bear more than their share of suffering. The HIV epidemic is no exception. You can continue to pursue social justice through your humane care and advocacy for disadvantaged patients. Finally, know your history. I find it enlivening to consider the sweep of the HIV epidemic as I care for my PLWH. What was it like for me to care for a similar patient 10, 20, or 30 years ago? I try to find meditative moments to express gratitude to the courageous physicians, advocates and patients, who struggled and sacrificed in the early days. My memory fades, of course, but the most incidental remark, image or even music brings it back in a technicolour Proustian moment. This gives perspective and meaning to my work. Coming of age now, you missed this often difficult but ultimately triumphant history. So, find a seasoned doctor who has worked in HIV from the beginning. Her stories will illuminate your path. Acknowledgements Conflicts of interest I offer this fictitious ‘letter to a young HIV doctor’ to counter a prevailing narrative that discourages physicians from pursuing or remaining in HIV medicine. There are no conflicts of interest.

Background There is an anticipated shortage of primary care providers trained to care for patients with HIV. The Yale School of Medicine developed and implemented a novel HIV training track within our Primary Care Internal Medicine Residency Program. A set of 12 Entrustable Professional Activities (EPAs) were developed to guide curriculum development and resident assessment. Aim To describe the process of implementing a novel EPA-based curriculum for the HIV Primary Care Training Track including EPA-based trainee evaluation tools. Participants/Settings Two to three residents were enrolled annually from 2012 to 2017 (total n = 11). Training sites included the outpatient academic center HIV clinic and inpatient HIV ward. Program Description An expert panel developed 12 HIV-specific EPAs. These were mapped to curricular and reporting internal medicine milestones. Curricular activities and evaluation tools were developed to guide EPA progress. Program Evaluation Graduating residents were ready for unsupervised practice in 91% of EPAs at the end of the 3-year program. Discussion Development of HIV-specific training EPAs was effective for driving curricular development and resident evaluation, and served as an effective method to communicate expectations to resident participants. These HIV-specific EPAs could serve as a useful template to enhance HIV education in academic settings.

Introduction Effective feedback is essential for trainees to achieve clinical competence. An unsupportive institutional feedback culture can diminish the credibility and subsequent uptake of feedback. Data on feedback culture in pediatric training programs is limited. We, therefore, used a validated feedback environment survey (FES) tool to assess and compare the cultures in our pediatric residency and fellowship programs. Methods Pediatric fellows and residents at Yale New Haven Hospital were invited to anonymously complete the FES using Qualtrics. The FES has a 7-point Likert scale response format, measures various aspects of the feedback environment, and has demonstrated good reliability, internal structure and relationship to other variables validity in an industrial setting. Fellow and resident ratings were compared using two-sided Fisher's exact test. Results Fifty-two (65%) residents and 21 (47%) fellows completed the survey. Most trainees reported that they respect their attendings’ opinion (76.7%), feel that they are fair (67.1%) and have confidence in their feedback (64.3%). However, the majority felt that the feedback they received was not useful (60.2%) and that attendings did not encourage them to ask for feedback (67.1%). When compared to fellows, more residents felt that attendings were unfamiliar with their performance (61.5% vs. 14.3%, p<0.001), not always available to provide feedback (57.7% vs. 25.0%, p=0.018) and did not give helpful feedback (78.8% vs. 45.0%, p=0.009). In addition, more residents reported that the only time they received feedback was through end-of-rotation evaluation forms (70.6% vs. 30.0%, p=0.003). Conclusions We have demonstrated deficiencies in the feedback culture of our pediatric residency and fellowship programs, which are more pronounced in the residency program. Identification of the factors contributing to the difference is important and will inform targeted interventions.

PROBLEM People with HIV/AIDS are living longer and are at an increased risk of comorbidities. A qualified physician workforce is needed to care for this growing population. APPROACH In 2012, a novel three-year HIV training track (HIV TT) was implemented as part of the Yale Primary Care Residency Program. To prepare for the implementation of this program, a needs assessment was performed, a web-based curriculum and 12 HIV-specific entrustable professional activities (EPAs) were created, and adequate clinical training opportunities in HIV and primary care were established. Program evaluation included process, learner, and outcome evaluations from 2012 to 2017. OUTCOMES Since its inception, the HIV TT has enrolled a total of 11 residents (6-7 at a time), with 5 graduating to date. Residents delivered high-quality HIV and primary care for a diverse panel of patients; improved their knowledge and performance in HIV care, including according to the HIV-specific EPAs; and were highly satisfied with the program. All faculty remained with the program, and patients indicated satisfaction. NEXT STEPS Next steps include enhanced coordination of residents' schedules, improved EPA documentation, evaluation of residents' HIV and non-HIV competence beyond residency, and monitoring graduates' career trajectories. Expanding HIV training within internal medicine residency programs is feasible and effective and has the potential to alleviate the shortage of physicians trained to provide HIV care and primary care in a single setting.

Background: Cognitive psychology studies demonstrate that subjects who attempt to recall information show better learning, retention, and transfer than subjects who spend the same time studying the same material (test-enhanced learning, TEL). We systematically reviewed TEL interventions in health professions education. Methods: We searched 13 databases, 14 medical education journals, and reference lists. Inclusion criteria included controlled studies of TEL that compared TEL to studying the same material or to a different TEL strategy. Two raters screened articles for inclusion, abstracted information, determined quality scores, and calculated the standardized mean difference (SMD) for the learning outcomes. Results: Inter-rater agreement was excellent for all comparisons. The 19 included studies reported 41 outcomes with data sufficient to determine a SMD. TEL interventions included short answer questions, multiple choice questions, simulation, and standardized patients. Five of six immediate learning outcomes (SMD 0.09-0.44), 21 of 23 retention outcomes (SMD 0.12-2.5), and all seven transfer outcomes (SMD 0.33-1.1) favored TEL over studying. Conclusions: TEL demonstrates robust effects across health professions, learners, TEL formats, and learning outcomes. The effectiveness of TEL extends beyond knowledge assessed by examinations to clinical applications. Educators should include TEL in health professions curricula to enhance recall, retention, and transfer.

Abstract Background Advances in HIV treatment changed the landscape of the epidemic from a fatal to a chronic disease. The number of patients living with HIV is expected to increase as they are living longer. Compared with the general population, older HIV-infected patients suffer additional comorbidities and often take several medications, leading to polypharmacy and drug interactions. Besides that the HIV population is aging, more patients know their status or want to access pre-exposure prophylaxis for prevention. Furthermore, the HIV workforce is aging and retiring without a new generation of providers to replace them. There is a fundamental concern about the readiness of future physicians to care for the HIV population. In response to this anticipated workforce shortage, an HIV Training Track was established at Yale Primary Care Residency Program in 2012. Methods Two to three residents were recruited into the HIV training track each year. Residents have their continuity practice in the institution’s HIV Clinic and rotate on the inpatient HIV Firm each year. Otherwise, residents participate in all of the core rotation and curricular activities of the Primary Care Residency. The authors will (i) display the process of building the infrastructure of the HIV training program, (ii) describe the curriculum, and (iii) share the 5 years experiences and outcomes. Results The program enrolled a cohort of 11 residents between 2012 and 2017. Residents managed a panel of 30–40 HIV-patients with diverse demographics. A medical record review revealed high performance measures in HIV and non-HIV conditions. 100% of eligible patients were on ART, 92% of patients were retained in care and 92% of those on ART had HIV viral suppression. In addition, all residents completed an HIV knowledge assessment test and showed 26% increase in their score at 1 year. There was 100% retention of residents and faculty. Residents and patients demonstrated high satisfaction with the program. Conclusion A novel HIV training track is feasible and can be successfully implemented. Expanding HIV-specific curricula within primary care residency program can build workforce of providers to meaningfully care for the aging HIV population. Disclosures All authors: No reported disclosures.

Entrustable professional activities (EPAs) have emerged as a construct to operationalize competency-based medical education. In a programmatic pilot, we assessed the feasibility and effectiveness of faculty coaches for assessing EPAs and facilitating reflective learning in medical students.

In 2014, the Association of American Medical Colleges (AAMC) published a list of 13 Core Entrustable Professional Activities for Entering Residency (Core EPAs) that medical school graduates might be expected to perform, without direct supervision, on the first day of residency. Soon after, the AAMC commissioned a five-year pilot with 10 medical schools across the United States, seeking to implement the Core EPA framework to improve the transition from undergraduate to graduate medical education.In this article, the pilot team presents the organizational structure and early results of collaborative efforts to provide guidance to other institutions planning to implement the Core EPA framework. They describe the aims, timeline, and organization of the pilot as well as findings to date regarding the concepts of entrustment, assessment, curriculum development, and faculty development. On the basis of their experiences over the first two years of the pilot, the authors offer a set of guiding principles for institutions intending to implement the Core EPA framework. They also discuss the impact of the pilot, its limitations, and next steps, as well as how the pilot team is engaging the broader medical education community. They encourage ongoing communication across institutions to capitalize on the expertise of educators to tackle challenges related to the implementation of this novel approach and to generate common national standards for entrustment. The Core EPA pilot aims to better prepare medical school graduates for their professional duties at the beginning of residency with the ultimate goal of improving patient care.