Introduction: Competency-Based Health Professions Education (CBHPE) is an outcomes-based approach to planning and conceptualizing the training of health professionals to serve population needs. Despite its uptake, ongoing debate persists regarding the assumptions underpinning CBHPE, its intended purposes, and how success should be defined and evaluated. This contributes to variable implementation and difficulty aligning core features with meaningful outcomes. To date, no published theory of change has articulated how CBHPE’s core components provide a framework for later evaluation of intended outcomes.Methods: Using Funnell and Rogers’ purposeful program theory, the authors, all members of the International Competency-based Education Health Professions Educators Collaborative (ICBHPE), developed a model theory of change. The authors conducted a broad review of CBHPE literature, supplemented by CBE frameworks from adjacent educational contexts to strengthen underdeveloped constructs in implementation and evaluation. Findings were iteratively synthesized into a program theory and refined through repeated cycles of large-group feedback, consensus building, and member checking within ICBHPE.Results: The authors developed a CBHPE Outcomes Theory of Change that articulates key assumptions, mechanisms of impact, and expected outcomes associated with CBHPE implementation. The framework specifies how van Melle’s CBHPE core components are theorized to function across phases of implementation and identifies proximal and distal outcomes that can reasonably be expected, based on intentional, deliberate program evaluation at each implementation and evaluation stage.Discussion: The CBHPE Outcomes Theory of Change offers scholars and implementers a practical guide to enhance the quality and fidelity of implementation, evaluation, and research efforts related to CBHPE worldwide. Using this guide as a framework, programs can plan, implement, and then examine whether the intended outcomes of CBHPE have been realized.

PURPOSE:Despite the movement toward competency-based medical education (CBME), most training programs in the United States still advance learners based on time-in-training rather than achievement of competency standards. The Education in Pediatrics Across the Continuum (EPAC) project sought to answer whether it is possible to make time-variable, competency-based advancement decisions by creating a program that spans undergraduate (UME) to graduate medical education (GME) to fellowship/practice. This article reports the outcomes of the EPAC project. METHOD:Each participating site (Universities of California-San Francisco, Colorado, Minnesota, and Utah) selected 4 consecutive cohorts and prospectively followed them from July 2013 to June 2023. Implementation allowed for site-variability but adhered to common design principles. The Core Entrustable Professional Activities (EPAs) for Entering Residency and the General Pediatrics EPAs were the competency frameworks. Each site convened a clinical competency committee every three to six months to determine readiness for learner transition from UME to GME and GME to practice/fellowship based on demonstrated competence. RESULTS:Fifty-six learners enrolled in EPAC and forty-four (79%) completed the program as designed. The time required for each learner to demonstrate readiness for transition from UME to GME (mean 20.2 four-week blocks [SD 2.76], range 14.5-26) and GME to practice/fellowship (mean 5.8 six-month blocks, [SD 0.34], range 4.9-6.1) varied. EPAC participants' educational outcomes were comparable to non-EPAC graduates at the same site, including Milestones ratings (B = 0.30, P = .21), first time pass rates for the American Board of Pediatrics initial certifying exam (93%, 95% CI 81%-98%), and attainment of fellowship or job placements after graduation. CONCLUSIONS:The outcomes of the EPAC project demonstrated the ability to make competency-based readiness-to-transition decisions for individual learners and is the first CBME program to span the UME/GME medical education continuum in the US. Lessons learned from the EPAC project have the potential to contribute to advancement efforts for CBME.

Competency-based education (CBE) in the health professions is grounded in teaching and assessing the requisite competencies to develop professionals prepared to meet the health care needs of the public and provide high-quality and safe patient care. As such, the outcome of education is the demonstrable competence of graduates in patient care. A curriculum and assessment framework based on entrustable professional activities (EPAs) provides excellent scaffolding for ensuring this outcome. The system is dependent, however, on its ability to support grounded, credible, and summative decisions regarding granting or withholding trainee progression through a program and awarding of increased autonomy (i.e., less supervision) in patient care. In this chapter, we begin by defining a clinical competency committee (CCC) and establishing its roles. We then provide a rationale for the group structure of a CCC as best suited to make the grounded, credible, and summative decisions required in an EPA-based curriculum and assessment system. Next, we explore more deeply the central role of a CCC—‘what’ it does. Then we address the ‘how’ of running a CCC, that is, the strategies to help CCCs function optimally, including issues of CCC structure and process. Finally, we explore some of the common pitfalls, misconceptions, and limitations regarding CCCs and suggest some mitigating strategies to overcome them.

Competency-based education of health professionals has been gaining momentum across the globe for the past two decades. The central tenet is to start with the outcomes that are required of a trainee to meet the health care needs of the public. These outcomes lead to the elaboration of requisite competencies, which in turn drive the curriculum and assessment programs. Educators have encountered many challenges in the development and implementation of curricula and assessment systems that effectively teach and measure performance of the requisite competencies. Entrustable professional activities (EPAs) offer an excellent framework for addressing many of those challenges. In this chapter, we offer the unique potential of an EPA framework in promoting a competency-based curriculum. We review some of the curricular structural implications of using EPAs, including the role of sequencing of EPAs in the curriculum, describe how an EPA-based curriculum adds agency to the trainee’s journey, and explore the notion of time-variability in competency-based education and training based on success stories using an EPA framework. We conclude that a well-designed curriculum and assessment system using EPAs provides an excellent foundation for ensuring health professionals’ readiness to provide safe and effective care within the scope of their discipline and at the appropriate level of supervision.

PURPOSE:This study examines the feasibility and psychometric results of an assessment of entrustable professional activities (EPAs) as a core component of the clinical program of assessment in undergraduate medical education, assesses the learning curves for each EPA, explores the time to entrustment, and investigates the dependability of the EPA data based on generalizability theory (G theory) analysis. METHOD:Third-year medical students from the University of Minnesota Medical School in 7 required clerkships from May 2022 through April 2023 were assessed. Students were required to obtain at least 4 EPA assessments per week on average from clinical faculty, residents supervising the students, or assessment and coaching experts. Student ratings were depicted as curves describing their performance over time; regression models were used to fit the curves. RESULTS:The complete class of 240 (138 women [58.0%] and 102 men [42.0%]) third-year medical students at the University of Minnesota Medical School (mean [SD] age at matriculation, 24.2 [2.7] years) participated. There were 32,614 EPA-based assessments (mean [SD], 136 [29.6] assessments per student). Reliability analysis using G theory found that an overall score dependability of 0.75 (range, 0-1) was achieved with 4 assessors on 4 occasions. The desired level of entrustment by academic year end was met by all 240 students (100%) for EPAs 1, 6, and 7, 237 (98.8%), 236 (98.3%), and 218 (90.8%) students for EPAs 2, 5, and 9, respectively, 197 students (82.1%) for EPA 3, 178 students (74.2%) for EPA 4, and 145 students (60.4%) for EPA 12. The most rapid growth was for EPA 2 (β 0 = .286), followed by EPA 1 (β 0 = .240), EPA 4 (β 0 = .236), and EPA 10 (β 0 = .230). CONCLUSIONS:The study findings suggest that EPA ratings provide reliable and dependable data to make entrustment decisions about students' performance.

Purpose To explore validity evidence for the use of entrustable professional activities (EPAs) as an assessment framework in medical education. Method Formative assessments on the 13 Core EPAs for entering residency were collected for 4 cohorts of students over a 9- to 12-month longitudinal integrated clerkship as part of the Education in Pediatrics Across the Continuum pilot at the University of Minnesota Medical School. The students requested assessments from clinical supervisors based on direct observation while engaging in patient care together. Based on each observation, the faculty member rated the student on a 9-point scale corresponding to levels of supervision required. Six EPAs were included in the present analyses. Student ratings were depicted as curves describing their performance over time; regression models were employed to fit the curves. The unit of analyses for the learning curves was observations rather than individual students. Results (1) Frequent assessments on EPAs provided a developmental picture of competence consistent with the negative exponential learning curve theory; (2) This finding was true across a variety of EPAs and across students; and (3) The time to attain the threshold level of performance on the EPA for entrustment varied by student and EPA. Conclusions The results provide validity evidence for an EPA-based program of assessment. Students assessed using multiple observations performing the Core EPAs for entering residency demonstrate classic developmental progression toward the desired level of competence resulting in entrustment decisions. Future work with larger data samples will allow further psychometric analyses of assessment of EPAs.

We thank Dr. Balon for his comments on our article. His suggested strategy shortens undergraduate medical education (UME) by reducing or “abolishing” the college education prerequisite for those planning a career in medicine. We acknowledge such an approach to medical education exists around the world, as well as in the United States. 1 A merged undergraduate and medical educational experience for individuals who are prepared to make an early career commitment to medicine omits more of what is traditionally viewed as the “liberal arts” curriculum. This arguably truncates educational life experiences that may occur during what are the more formative adult years for some students. One result of such “earlier” accelerated programs might be to narrow the breadth of life that individuals bring to the medical profession, which may reduce cultural and experiential diversity of the resulting cadre of future physicians. Nevertheless, we acknowledge that such a pathway may be appropriate for some students provided they have the time and experience before medical school to develop the requisite competencies. 2 Alternatively, we have suggested an acceleration of educational advancement after the primary decision has been made to pursue a career in medicine. The net effect of this approach is to move the medical specialization decision, rather than the decision to pursue a medical career, to an earlier point in the education continuum. We believe this approach allows for more formative experiences before the commitment to a medical career, hence supporting more personal growth of the individual. Additionally, from a more pragmatic standpoint, placing the acceleration point in medical school puts the implementation of this transition in the hands of relevant stakeholders who already have interest in both components of medical education. Strategically, this facilitates the trainee staying in a single institution for both UME and graduate medical education (GME) components, which provides greater continuity of education and arguably enhances the quality and oversight of the accelerated experience for both the learner and the program. In summary, a host of options exist to accelerate medical education and only further experience will demonstrate which are most suitable for each individual trainee. 3 Any program that decreases time to graduation must be tied to competence, which is more suited to an assessment of GME mastery than GME preparation. An open-minded approach and careful study of outcomes will be critical to defining the most attractive and successful programs.

The international movement to competency-based medical education (CBME) marks a major transition in medical education that requires a shift in educators' and learners' approach to clinical experiences, the way assessment data are collected and integrated, and in learners' mindsets. Learners entering a CBME curriculum must actively drive their learning experiences and education goals. For some, this expectation may be a significant change from their previous approach to learning in medicine. This paper highlights 12 tips to help learners succeed within a CBME model.

Problem Assessment has been the Achilles heel of competency-based medical education. It requires a program of assessment in which outcomes are clearly defined, students know where they are in the development of the competencies, and what the next steps are to attaining them. Achieving this goal in a feasible manner has been elusive with traditional assessment methods alone. The Education in Pediatrics Across the Continuum (EPAC) program at the University of Minnesota developed a robust program of assessment that has utility and recognizes when students are ready for the undergraduate to graduate medical education transition. Approach The authors developed a learner-driven program of assessment in the foundational clinical training of medical students in the EPAC program based on the Core Entrustable Professional Activities for Entering Residency (Core EPAs). Frequent workplace-based assessments, coupled with summative assessments, informed a quarterly clinical competency committee and individualized learning plans. The data were displayed on real time dashboards for the students to review. Outcomes Over 4 cohorts from 2015 to 2019, students (n = 13) averaged approximately 200 discrete Core EPA workplace-based assessments during their foundational clinical training year. Assessments were completed by an average of 9 different preceptors each month across 8 different specialties. The data were displayed in a way students and faculty could monitor development and inform a clinical competency committee’s ability to determine readiness to transition to advanced clinical rotations and residency. Next Steps The next steps include continuing to scale the program of assessment to a larger cohort of students.

Eva Aagaard, MD Cary Aarons, MD Sawsan Abdel-Razig, MD Mark Abrams, MD Rolf Ahlzen, MD Ralitsa Akins, MD Monica Alba Sandoval, MD Ibrahim Al-Busaidi, MB Suzanne Allen, MD Al’ai Alvarez, MD Meshari Alwashmi, PhD Richard Alweis, MD Ashley Amick, MD Douglas Ander, MD Hannah Anderson, MBA Marsha Anderson, MD John Andrews, MD Dorothy Andriole, MD Tanya Anim, MD Tyler Anstett, DO Thurayya Arayssi, MD Chandrakanth Are, MD Vineet Arora, MD Cherinne Arundel, MD Amber Atwater, MD Marc Auerbach, MD Alex Auseon, DO Yelena Averbukh, MD C. Maria Bacchus, MD Colleen Badke, MD Laura Baecher-Lind, MD Arianne Baker, MD Tessa Balach, MD Miriam Bar-on, MD Maneesh Batra, MD Michael Battistone, MD Jimmy Beck, MD Kimberly Becker, PhD Kristen Bene, PhD Jeffrey Bennett, MD William Benton, MD Roseanne Berger, MD Abiona Berkeley, MD Jeffrey Berns, MD Carol Bernstein, MD Joanne Bernstein, MD Kelly Best, MD Mark Best, MD Beth Bierer, PhD Corey Bills, MD Robert Bing-You, MD Rebecca Blanchard, PhD Thomas Bodenheimer, MD Christy Boscardin, PhD Pierre Bou Khalil, MD Donald Boyer, MD Megan Boysen-Osborn, MD Joseph Bradley, MD Jeremy Branzetti, MD Daniel Breitkopf, MD Fred Buckhold, MD Ann Burke, MD John Burkhardt, MD Roger Bush, MD Andrew Butler, PhD William Bynum, MD Aaron Calhoun, MD Nora Callinan, MD Jason Campbell, MD Donna Caniano, MD Holly Caretta-Weyer, MD Christopher Carroll, MD Sebastiano Cassaro, MD Tara Catanzano, MD Rodrigo Cavalcante, MD Carlyle Chan, MD Ravi Chandra, MD Timothy Chaplin, MD Xiaodong (Phoenix) Chen, PhD Fei Chen, PhD Jeffrey Cheung, PhD Davoren Chick, MD Nicole Chiota-McCollum, MD Nicole Christian, MD Colleen Christmas, MD Saumil Chudgar, MD Koong-Nah Chung, PhD Thomas Ciesielski, MD Michelle Clarke, MD Alison Clay, MD

Construct: We investigated whether a situational judgment test (SJT) designed to measure professionalism in physicians predicts residents' performance on (a) Accreditation Council for Graduate Medical Education (ACGME) competencies and (b) a multisource professionalism assessment (MPA). Background: There is a consensus regarding the importance of assessing professionalism and interpersonal and communication skills in medical students, residents, and practicing physicians. Nonetheless, these noncognitive competencies are not well measured during medical education selection processes. One promising method for measuring these noncognitive competencies is the SJT. In a typical SJT, respondents are presented with written or video-based scenarios and asked to make choices from a set of alternative courses of action. Interpersonally oriented SJTs are commonly used for selection to medical schools in the United Kingdom and Belgium and for postgraduate selection of trainees to medical practice in Belgium, Singapore, Canada, and Australia. However, despite international evidence suggesting that SJTs are useful predictors of in-training performance, end-of-training performance, supervisory ratings of performance, and clinical skills licensing objective structured clinical examinations, the use of interpersonally oriented SJTs in residency settings in the United States has been infrequently investigated. The purpose of this study was to investigate whether residents' performance on an SJT designed to measure professionalism-related competencies-conscientiousness, integrity, accountability, aspiring to excellence, teamwork, stress tolerance, and patient-centered care-predicts both their current and future performance as residents on two important but conceptually distinct criteria: ACGME competencies and the MPA. Approach: We developed an SJT to measure seven dimensions of professionalism. During calendar year 2017, 21 residency programs from 2 institutions administered the SJT. We conducted analyses to determine the validity of SJT and USMLE scores in predicting milestone performance in ACGME core competency domains and the MPA in June 2017 and 3 months later in September 2017 for the MPA and 1 year later, in June 2018, for ACGME domains. Results: At both periods, the SJT score predicted overall ACGME milestone performance (r = .13 and .17, respectively; p < .05) and MPA performance (r = .19 and .21, respectively; p < .05). In addition, the SJT predicted ACGME patient care, systems-based practice, practice-based learning and improvement, interpersonal and communication skills, and professionalism competencies (r = .16, .15, .15, .17, and .16, respectively; p < .05) 1 year later. The SJT score contributed incremental validity over USMLE scores in predicting overall ACGME milestone performance (ΔR = .07) 1 year later and MPA performance (ΔR = .05) 3 months later. Conclusions: SJTs show promise as a method for assessing noncognitive attributes in residency program applicants. The SJT's incremental validity to the USMLE series in this study underscores the importance of moving beyond these standardized tests to a more holistic review of candidates that includes both cognitive and noncognitive measures.

Purpose To determine which narrative performance level for each general pediatrics entrustable professional activity (EPA) reflects the minimum level clinical competency committees (CCCs) felt should be associated with graduation as well as initial entrustment and compare expected narrative performance levels (ENPLs) for each EPA with actual narrative performance levels (ANPLs) assigned to residents at initial entrustment. Method A series of 5 narratives, corresponding to the 5 milestone performance levels, were developed for each of the 17 general pediatrics EPAs. In academic year (AY) 2015-2016, the CCCs at 22 Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network member sites reported ENPLs for initial entrustment and at time of graduation. From AYs 2015-2016 to 2017-2018, programs reported ANPLs for initial entrustment decisions. ENPLs and ANPLs were compared using a logistic mixed effects model. Results ENPLs for graduation and entrustment were most often level 3 (competent) followed by level 4 (proficient). For 8 EPAs, the ENPLs for graduation and entrustment were the same. For the remaining 9, some programs would entrust residents before graduation or graduate them before entrusting them. There were 4,266 supervision level reports for initial entrustment for which an ANPL was provided. ANPLs that were lower than the ENPLs were significantly more likely to be assigned to the medical home-well child (OR = 0.39; 95% CI: 0.26-0.57), transition to adult care (OR = 0.43; 95% CI: 0.19-0.95), behavioral or mental health (OR = 0.36; 95% CI: 0.18-0.71), make referrals (OR = 0.31; 95% CI: 0.17-0.55), lead a team (OR = 0.34; 95% CI: 0.22-0.52), and handovers (OR = 0.18; 95% CI: 0.09-0.36) EPAs. Conclusions CCCs reported lower ENPLs for graduation than for entrustment for 5 EPAs, possibly indicating curricular gaps that milestones and EPAs could help identify.

Purpose To determine whether longitudinal student involvement improves patient satisfaction with care. Method The authors conducted a satisfaction survey of patients followed by 10 University of Minnesota Medical School students enrolled in 2016-2017 in the Veterans Affairs Longitudinal Undergraduate Medical Education (VALUE) program, a longitudinal integrated clerkship at the Minneapolis Veterans Health Care System. Students were embedded in an ambulatory practice with primary preceptors who assigned students a panel of 14 to 32 patients to follow longitudinally in inpatient and outpatient settings. Control patients, matched on disease severity, were chosen from the preceptor's panel. Two to five months after the students completed the VALUE program, the authors conducted a phone survey of the VALUE and control patients using a validated, customized questionnaire. Results Results are reported from 97 VALUE patients (63% response rate) and 72 controls (47% response rate) who had similar baseline characteristics. Compared with control patients, VALUE patients reported greater satisfaction with explanations provided by their health care provider, their provider's knowledge of their personal history, and their provider's looking out for their best interests (P < .05). Patients in the VALUE panel selected the top category more often than control patients for overall satisfaction with their health care (65% vs 43%, P < .05). Conclusions The results of this controlled trial demonstrate that VALUE student longitudinal participation in patient care improves patient satisfaction and patient-perceived quality of health care for VALUE patients compared with controls matched by primary care provider and disease severity. These findings may have implications outside the Veterans Administration population.

Medical Education Program Highlights Founded in 1888, the University of Minnesota Medical School (UMMS) is 1 of the 10 largest medical schools in the country—with 240 students per class across 2 campuses in the Twin Cities and Duluth. As a land-grant institution, we are dedicated to improving the health of the state’s citizens and beyond, educating its future physicians, and creating new knowledge through discovery. It is estimated that 70% of the state’s physician workforce is trained at UMMS, either in UME, GME, or both. Mullan and colleagues defined a social mission score and ranked the nation’s 141 MD-granting medical schools using that score.1 In their 2010 publication, UMMS was 1 of only 2 schools in the country to rank in the top quartile for social mission, NIH funding, and primary care output. Our social mission ranking aligns with our UME vision: “A community, learning together, to prepare exceptional physicians to improve the health and well-being of Minnesota and beyond.” To achieve that vision, we have delineated 7 guiding principles: Build on diversity and inclusion Empower students Put patients first Standardize the outcomes, individualize the learning pathways Foster relationships Optimize the learning environment Provide evidence-based education These guiding principles are the foundation for UME and are threaded through the following highlights: Individualized pathways with the opportunity to focus on rural health, research, earn a dual degree in our FLEX MD program, study abroad, or complete core clerkships in 1 of our 6 unique longitudinal integrated clerkships (LICs). We are 1 of 4 schools participating in Education for Pediatrics Across the Continuum. The program allows for competency-based, time-variable progression from medical school to pediatrics residency. In 2020, we move into the new Health Sciences Education Center. This state-of-the-art facility is designed to transform health education in Minnesota. The learning environment supports interprofessional education and active learning pedagogies, along with a focus on simulation and emerging technologies including VR/AR. Interprofessional student lounges and study and wellness spaces are also prominent features. Our regional campus in Duluth has a specific mission in educating physicians dedicated to family medicine to serve the needs of rural Minnesota and Native American communities and is a perennial leader in this area. We focus on student, staff, and faculty well-being: engaging our community through innovative programming and tools to incorporate well-being into daily practice. Curriculum Curriculum description Preclerkship years: This phase is organized into 2 academic years and offered in unique formats at the 2 campuses. Each campus provides students with a strong preclinical sequence that ensures essential foundational sciences are covered. Both campuses include additional specialized content areas shaped by the needs of the campus. Content spanning the preclerkship years includes rural medicine education and experiences as well as Essentials of Clinical Medicine and Foundations of Critical Thinking courses. Regional campus students transition to the Twin Cities campus for the final 2 years of clinical training. Year 3: Students select to complete their required foundational clerkships in a block or an LIC format during the third year. Students who choose block clerkships rotate in sequenced lanes and with a cohort of students through their required clerkships. Year 4: In this time frame, students complete 2 advanced required clerkships (emergency medicine and intensive care subinternship), 2 advanced selective clerkships, and at least 20 weeks of electives. Curriculum changes since 2010 The curriculum has evolved over the last decade. Key changes are: Intentional integration of basic and clinical knowledge in the foundational science courses Greater emphasis on health system science, health policy, and population health Expansion of longitudinal clinical experiences, including 4 new LICs More individualization of clinical training in year 4 to enhance preparation for residency (advanced care selectives based on future specialty) “Return to school” weeks during the clinical phase of training in our new Becoming a Doctor course; this course includes health system science, medical socioeconomics, and pain management and offers an opportunity for students to propose workshop topics See Table 1—Current Longitudinal Integrated Clerkships.Table 1: Current Longitudinal Integrated ClerkshipsClass size changes since 2010 The class size was increased from 165 to 175 on the Twin Cities Campus, and from 60 to 65 on the Duluth regional campus. The rationale was based on projections of future state workforce needs especially for rural and primary care providers and increased classroom and clinical training capacity. Assessment UMMS adopted the AAMC Physician Competency Reference Set (PCRS) as its educational program objectives, adding 2 competencies under a ninth domain, scientific and clinical inquiry. The PCRS is founded in the ACGME and American Board of Medical Specialties Outcome Project competencies, thus providing continuity for students as they head into residency and practice. Assessment changes since 2010 Along with curricular changes, we have also created alignment and altered our assessment strategies in the following ways: Elimination of grades in the first 2 years moving, to a pass/fail system Standardization of assessment based on common course objectives in the preclinical phase, including integration of competency-based testing with robust psychometric analysis Development of a framework for assessment for direct observation in the clinical phase, focusing on the Core Entrustable Professional Activities for Entering Residency Increased use of peer assessment of students Implementation of “Pulse Surveys” in 2018 for real-time tracking of burnout and satisfaction Development of the Medical Education Outcomes Center (MEOC) to centralize all educational data, make data-driven decisions, and link educational to clinical outcomes Parallel curriculum or tracks We have 1 parallel track, comprising the Rural Physician Associate Program (RPAP)/Metropolitan Physician Associate Program (MetroPAP). These are 9-month, community-based LICs for third-year medical students. RPAP occurs in rural community training sites, and MetroPAP in medically underserved and diverse urban communities. A 3-year MD degree program does not exist at UMMS; however, we have the 7-year BA–MD Joint Admissions Scholars Program, providing an opportunity for exceptional Minnesota residents from broadly diverse backgrounds. Three years are spent taking undergraduate coursework and 4 years are spent at the medical school. Pedagogy The frequency that various instructional methods are employed during years 1 and 2, and years 3 and 4, is varied. A major focus has been to increase the amount of active learning in large-group settings. See Supplemental Digital Appendix 1—Sessions Using AAMC Instructional Methods in Year 1 and 2 Required Courses—at https://links.lww.com/ACADMED/A878. See Supplemental Digital Appendix 2—Sessions Using AAMC Instructional Methods in Year 3 and 4 Required Courses—at https://links.lww.com/ACADMED/A878. Clinical experiences Our inpatient clinical teaching sites at which medical students take 1 or more clinical clerkships include 14 hospitals in Minneapolis and St. Paul, 2 hospitals in Duluth, and 1 hospital in St. Cloud. For our RPAP program, 46 hospitals in Greater Minnesota are used. Multiple ambulatory clinical teaching sites are used, including hospital-associated clinics, community health centers, private physician offices, and rural clinics. The first clinical encounter for year 1 students on the Twin Cities campus occurs in the Process of Care clerkships where students have the opportunity to see patients in clinic, hospital, and extended care settings. On the Duluth campus, the Rural Medical Scholars Program in the first year places students in rural communities living and working with medical preceptors in a clinical environment. The major challenge is capacity for clinical training related to faculty effort, but being the only medical school in the Twin Cities is an advantage. Availability of housing at some sites remains a challenge. Curricular Governance The Education Council (EC) is the curriculum committee of UMMS and is composed of faculty, students, and educational leaders. EC is the approved body for final decisions in curriculum and policy. The committees that report to EC include the Education Steering Committee that serves as an advisory group (“think tank”), Scientific Foundations Committee (Twin Cities year 1 and 2 course directors), Committee on Undergraduate Medical Education Duluth (Duluth course directors), Clinical Education Committee (clerkship directors), and the Assessment Committee that oversees all aspects of assessment in the MD program. All aspects of curricular governance are managed centrally with funds allocated to departments to support faculty teaching. See Figure 1—Curriculum governance structure.Figure 1: Curriculum governance structure.Education Staff The Office of Medical Education (OME), directed by the vice dean for education and academic affairs, administers the medical education program and works with faculty to develop programming and strategy. OME comprises the following components and leaders to oversee the continuum of medical education: Admissions and pipeline programs (associate dean of admissions) UME (associate dean of UME) GME (associate dean and designated institute official) Continuing professional development (director of continuing professional development) Anatomy bequest program (director of anatomy bequest) Program in mortuary science (director) Medical educator development and scholarship (MEDS) (director) MEOC (vice dean for education and academic affairs) See Figure 2—Organizational chart.Figure 2: Organizational chart.Faculty Development and Support in Education While faculty appointments are in medical school departments and not in OME, the office does provide substantial faculty development and support. MEDS is charged with improving the quality and effectiveness of medical education by providing faculty with the skills and tools they need to be effective teachers, educational leaders, and scholars. Programming includes workshops, works-in-progress conference, annual best practice conference, individual consultations, web-based tool kit, and research support through the MEOC. The assistant dean for assessment and evaluation provides faculty with workshops and consultations designed to ensure sound teaching practices. These include but are not limited to improving the quality of narrative assessment, creating effective exam questions, and interpreting exam results using effective statistical methods. The assistant dean for curriculum and a team that includes 2 directors (director of integrated education in the foundational sciences and director of integrated education in the clinical sciences) work with faculty on all aspects of curriculum development. The University of Minnesota Center for Educational Innovation (CEI) supports faculty in identifying sound educational approaches for teaching. Faculty development with a member of the CEI team is a standing item on the monthly course directors meeting. One-on-one consultations with a CEI education specialist are available to provide pedagogical support. Role of teaching in promotion and tenure UMMS has 3 tracks for promotion: tenure track, academic track, and master clinician track. For promotion on the tenure track, educational scholarship is a requirement. For promotion on the academic track, candidates must demonstrate sustained peer-reviewed scholarship in the form of a traditional peer-reviewed publication or other products, including peer-reviewed online repositories. Other criteria include development of educational products that have been adopted by others outside of the institution; successful grant funding; instructional, curricular, or assessment innovations; educational policy development; or chapter or book authorship. Candidates must demonstrate excellence in their educational roles demonstrated by learner reviews, teaching awards, and peer evaluations. For promotion on the master clinician track, the major focus is clinical excellence, which is defined differently by departments. For most departments, demonstration of teaching excellence is required and must be documented by learner evaluations. The University of Minnesota has the Academy for Excellence in the Scholarship of Teaching and Learning, which is open to all health science schools. The academy recognizes faculty who have demonstrated exceptional scholarly contributions to advance learning in their schools and across academic programs. Regional Medical Campuses UMMS Duluth was funded by the legislature in 1969, and the first students matriculated in 1972. Integration of the Twin Cities and Duluth campuses occurred in 2004. The Duluth campus provides the preclinical phase of medical education, after which the students enter the clinical phase of training at the Twin Cities campus. The mission of the Duluth campus is focused on rural medicine and serving Native American communities. A comparable educational experience is demonstrated by common learning objectives and assessments and standardized national examinations. Reporting relationships and regularly scheduled interactions between educational leaders, curriculum committees, and course directors at both campuses facilitate operational issues regarding comparability.

Purpose To evaluate response process validity evidence for clinical competency committee (CCC) assessments of first-year residents on a subset of General Pediatrics Entrustable Professional Activities (EPAs) and milestones in the context of a national pilot of competency-based, time-variable (CBTV) advancement from undergraduate to graduate medical education. Method Assessments of 2 EPAs and 8 milestones made by the trainees’ actual CCCs and 2 different blinded “virtual” CCCs for 48 first-year pediatrics residents at 4 residency programs between 2016 and 2018 were compared. Residents had 3 different training paths from medical school to residency: time-variable graduation at the same institution as their residency, time-fixed graduation at the same institution, or time-fixed graduation from a different institution. Assessments were compared using ordinal mixed-effects models. Results Actual CCCs assigned residents higher scores than virtual CCCs on milestones and one EPA’s supervision levels. Residents who graduated from a different institution than their residency received lower milestone ratings than either group from the same institution; CBTV residents received higher ratings on one milestone (ICS4) and similar ratings on all others compared with non-CBTV residents who completed medical school at the same institution. Conclusions First-year residents who graduated from CBTV medical school programs were assessed as having the same level of competence as residents who graduated from traditional medical school programs, but response process evidence suggests that members of CCCs may also draw on undocumented personal knowledge of the learner to draw conclusions about resident competence.

Question What is the progression of performance for entrustable professional activities (EPAs) throughout pediatric residency training and at graduation? Findings This multisite cohort study of 1987 pediatric residents found that developmental growth curves can be established for EPAs. When generated to reflect the results in this study, at least 90% of trainees achieved the level of unsupervised practice at the end of residency for only 8 of the 17 EPAs studied. Meaning This study suggests that gaps exist between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve based on the general pediatrics EPAs. Importance Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. Objective To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. Design, Setting, and Participants Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. Interventions Longitudinal, prospective assessment using EPAs. Main Outcomes and Measures Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. Results Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). Conclusions and Relevance This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence. This cohort study measures the development of pediatric resident clinical skills using assessments based on entrustable professional activities.

In 2016, Batalden et al proposed a coproduction model for health care services. Starting from the argument that health care services should demonstrate service-dominant rather than goods-dominant logic, they argued that health care outcomes are the result of the intricate interaction of the provider and patient in concert with the system, community, and, ultimately, society. The key notion is that the patient is as much an expert in determining outcomes as the provider, but with different expertise. Patients come to the table with expertise in their lived experiences and the context of their lives.The authors posit that education, like health care services, should follow a service-dominant logic. Like the relationship between patients and providers, the relationship between learner and teacher requires the integrated expertise of each nested in the context of their system, community, and society to optimize outcomes. The authors then argue that health professions learners cannot be educated in a traditional, paternalistic model of education and then expected to practice in a manner that prioritizes coproductive partnerships with colleagues, patients, and families. They stress the necessity of adapting the health care services coproduction model to health professions education. Instead of asking whether the coproduction model is possible in the current system, they argue that the current system is not sustainable and not producing the desired kind of clinicians.A current example from a longitudinal integrated clerkship highlights some possibilities with coproduced education. Finally, the authors offer some practical ways to begin changing from the traditional model. They thus provide a conceptual framework and ideas for practical implementation to move the educational model closer to the coproduction health care services model that many strive for and, through that alignment, to set the stage for improved health outcomes for all.

It is highly unusual for learners to leave medical training in the United States even though some individuals’ goals may change and others may not achieve expected competence. There are a number of possible reasons for this: (1) Students may feel that they have progressed too far into their careers and amassed too much debt to leave medical training; (2) students may be allowed to graduate despite marginal performance; and (3) students may have entered medical training with risk factors for poor performance that were not addressed. As stewards of the educational process, medical educators have an ethical obligation to students and the public to create off-ramps, or points along the educational continuum at which learners can reassess their goals and educators can assess competence, that allow students to leave medicine. Given the nationwide focus on physician health and wellness, the authors believe the creation of options to leave medical training without compromising one’s self-esteem or incurring unmanageable debt (i.e., compassionate off-ramps) is a moral imperative. The practice of medicine should not be an exercise in survival; it should allow people to develop and thrive over the course of their careers. Offering students options to make use of the medical competencies they have accumulated in other attractive careers would enable medical educators to behave compassionately toward individual students and fulfill their societal obligation to graduate competent and committed physicians. To this end, the authors present six recommendations for consideration.