Introduction: Competency-Based Health Professions Education (CBHPE) is an outcomes-based approach to planning and conceptualizing the training of health professionals to serve population needs. Despite its uptake, ongoing debate persists regarding the assumptions underpinning CBHPE, its intended purposes, and how success should be defined and evaluated. This contributes to variable implementation and difficulty aligning core features with meaningful outcomes. To date, no published theory of change has articulated how CBHPE’s core components provide a framework for later evaluation of intended outcomes.Methods: Using Funnell and Rogers’ purposeful program theory, the authors, all members of the International Competency-based Education Health Professions Educators Collaborative (ICBHPE), developed a model theory of change. The authors conducted a broad review of CBHPE literature, supplemented by CBE frameworks from adjacent educational contexts to strengthen underdeveloped constructs in implementation and evaluation. Findings were iteratively synthesized into a program theory and refined through repeated cycles of large-group feedback, consensus building, and member checking within ICBHPE.Results: The authors developed a CBHPE Outcomes Theory of Change that articulates key assumptions, mechanisms of impact, and expected outcomes associated with CBHPE implementation. The framework specifies how van Melle’s CBHPE core components are theorized to function across phases of implementation and identifies proximal and distal outcomes that can reasonably be expected, based on intentional, deliberate program evaluation at each implementation and evaluation stage.Discussion: The CBHPE Outcomes Theory of Change offers scholars and implementers a practical guide to enhance the quality and fidelity of implementation, evaluation, and research efforts related to CBHPE worldwide. Using this guide as a framework, programs can plan, implement, and then examine whether the intended outcomes of CBHPE have been realized.

Background Program directors need concrete indicators to monitor uptake of competency-based medical education (CBME). Entrustable professional activity (EPA) observation completion rates offer practical measures of CBME adoption. Objective In this study, we used residents' EPA observation data in clinical departments, specifically the submission and expiration of EPA observation forms and assessment scores, to explore the uptake of CBME practices across departments. Our research question asked: What are the patterns and contributing factors (department group, resident year, calendar year, program size) associated with EPA observation submission rates, expiration rates, and assessment scores? Methods We conducted exploratory analysis of de-identified EPA observation data (n=233 176) from residents' electronic portfolios (n=2110) across 45 programs in 12 departments at one Canadian institution from 2018 to 2023. Descriptive statistics summarized submission, expiration, and score distributions. Spearman correlations and logistic regression examined 4 predictors: department group, resident year, calendar year, and program size. Results EPA submission rates (81.0%), expiration rates (7.7%), and assessment O-scores (M=4.4 out of 5) did not differ significantly by training department. Calendar year increased odds of an independent or full score by 26.3% per year (OR, 1.263; 95% CI, 1.259-1.267) while resident year (OR, 0.818; 95% CI, 0.813-0.825) and program size (OR, 0.995; 95% CI, 0.994-0.996) decreased those odds. Conclusions EPA submission, expiration, and scoring patterns are consistent across departments and correlate with implementation year, resident training stage, and program size.

Background Competency-based medical education (CBME) has been implemented in many residency training programs across Canada. A key component of CBME is documentation of frequent low-stakes workplace-based assessments to track trainee progression over time. Critically, the quality of narrative feedback is imperative for trainees to accumulate a body of evidence of their progress. Suboptimal narrative feedback will challenge accurate decision-making, such as promotion to the next stage of training. Objective To explore the quality of documented feedback provided on workplace-based assessments by examining and scoring narrative comments using a published quality scoring framework. Methods We employed a retrospective cohort secondary analysis of existing data using a sample of 25% of entrustable professional activity (EPA) observations from trainee portfolios from 24 programs in one institution in Canada from July 2019 to June 2020. Statistical analyses explore the variance of scores between programs (Kruskal-Wallis rank sum test) and potential associations between program size, CBME launch year, and medical versus surgical specialties (Spearman's rho). Results Mean quality scores of 5681 narrative comments ranged from 2.0±1.2 to 3.4±1.4 out of 5 across programs. A significant and moderate difference in the quality of feedback across programs was identified (χ2=321.38, P<.001, ε2=0.06). Smaller programs and those with an earlier launch year performed better (P<.001). No significant difference was found in quality score when comparing surgical/procedural and medical programs that transitioned to CBME in this institution (P=.65). Conclusions This study illustrates the complexity of examining the quality of narrative comments provided to trainees through EPA assessments.

Small-group learning is a mainstay of medical education, and group functioning can have a major influence on these learning experiences. Our objective was to explore verbal exchange patterns within small-group learning sessions and examine how different patterns related to tutor involvement, tutor expertise, and participants’ perceptions. A non-participant observer collected group interactivity data using a real-time mobile device-based system. Verbal interaction patterns were visualized and analyzed using social network analysis and correlated with participant survey data and aggregate course grades. There were 46 observations across 30 separate groups. Group interactions clustered into four patterns defined by (1) tutor involvement (high vs. low) and (2) interactivity (high vs. low). Interaction patterns were largely stable for a given group and groups with content expert facilitators were generally less interactive. Students reported objectively fewer interactive groups as more interactive and enjoyable. There were no significant intergroup differences in aggregate course grades. Paradoxically, student perceptions were not aligned with observed interactivity data, and tutor content expertise influenced group interactivity. These findings suggest the need to better manage learner expectations of small-group learning, and to explicitly reflect on and develop skills for effective collaborative learning with both faculty and students.

In the past decade, the Canadian system of postgraduate medical education has been transformed with the implementation of a new approach to competency based medical education called Competence by Design. The Royal College of Physicians and Surgeons of Canada (Royal College) developed an approach to time-variable competency based medical education and adapted that design for medical, surgical, and diagnostic disciplines. New educational standards and entrustable professional activities consistent with this approach were co-created with 67 specialties and subspecialties, and implementation was scaled up across 17 universities and over 1000 postgraduate training programs. Partner engagement, systematic design of workshops to create discipline specific competency-based standards of education, and agile adaptation were all key ingredients for success. This paper describes the strategies applied by the Royal College, lessons learned regarding transformative change in the complex system of postgraduate medical education, and the current status of the Competence by Design initiative. The approach taken and lessons learned by the Royal College may be useful for other educators who are planning a transformation to CBME or any other major educational reform.

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Competency based medical education (CBME) is a global movement to reform health professions education (HPE).In contrast to the traditional time-based training, it is an educational design approach that emphasizes necessary learner abilities, greater learner-centeredness, and better alignment with public's needs for healthcare providers.Unlike numerous previous HPE innovations, CBME is framed as a complex transformational change with multiple proposed elements [1].Training programs employing a competency-based approach can have five core components to operationalize their implementation with fidelity: outcome competencies, progressive sequencing, tailored learning experiences, competency-focused instruction, and programmatic assessment [2].Health professions education institutions, disciplines, and programs across the globe have been implementing CBME at an ever-increasing rate.The CBME movement began with early innovations such as the CanMEDS framework [3-7], the ACGME Outcomes Project [8-10], entrustable professional activities [11,12], the Dutch CBME curriculum [13,14], the Cincinnati observable practice activities [15], the College of Family Physicians of Canada Triple C Curriculum [15-17], and the University of Toronto Orthopedic program [18].CBME has now spread worldwide with implementations in, for example, Switzerland, Taiwan, Australian Orthopedic training [19], Finland, South Africa, Brazil, and many more countries.Competency based education is now a part of many health professions beyond medicine, including nursing [20] and veterinary medicine [21].Despite the global implementation effort, the current discourse around CBME is still largely focused on theoretical underpinnings and potential [22,23] and countered with skeptical criticisms [24,25].There are early efforts to evaluate and clarify the outcomes of these CBME curricula [26,27].However, what is needed at this stage of the diffusion of these innovations is sharing CBME praxis: examples of the application of the CBME approach to real-life HPE systems and the lessons learned for others who follow.In this special issue of Perspectives on Medical Education, a team of educators has come together to describe a unique Canadian configuration of CBME implementation called Competence by Design (CBD).Worldwide, the majority of CBME innovations have focused on implementing

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Coaching is an increasingly popular means to provide individualized, learner-centered, developmental guidance to trainees in competency based medical education (CBME) curricula. Aligned with CBME's core components, coaching can assist in leveraging the full potential of this educational approach. With its focus on growth and improvement, coaching helps trainees develop clinical acumen and self-regulated learning skills. Developing a shared mental model for coaching in the medical education context is crucial to facilitate integration and subsequent evaluation of success. This paper describes the Royal College of Physicians and Surgeons of Canada's coaching model, one that is theory based, evidence informed, principle driven and iteratively and developed by a multidisciplinary team. The coaching model was specifically designed, fit for purpose to the postgraduate medical education (PGME) context and implemented as part of Competence by Design (CBD), a new competency based PGME program. This coaching model differentiates two coaching roles, which reflect different contexts in which postgraduate trainees learn and develop skills. Both roles are supported by the RX-OCR process: developing Relationship/Rapport, setting eXpectations, Observing, a Coaching conversation, and Recording/Reflecting. The CBD Coaching Model and its associated RX-OCR faculty development tool support the implementation of coaching in CBME. Coaching in the moment and coaching over time offer important mechanisms by which CBD brings value to trainees. For sustained change to occur and for learners and coaches to experience the model's intended benefits, ongoing professional development efforts are needed. Early post implementation reflections and lessons learned are provided.

Transformative changes in health professions education need to incorporate effective faculty development, but few very large-scale faculty development designs have been described. The Royal College of Physicians and Surgeons of Canada's Competence by Design project was launched to transform the delivery of postgraduate medical education in Canada using a competency-based model. In this paper we outline the goals, principles, and rationale of the Royal College's national strategy for faculty and resident development initiatives to support the implementation of Competence by Design. We describe the activities and resources for both faculty and trainees that facilitated the redesign of training programs for each specialty and subspecialty at the national level, as well as supporting the implementation of the redesign at the local level. This undertaking was not without its challenges: we thus reflect on those challenges, enablers, and the lessons learned, and discuss a continuous quality improvement approach that was taken to iteratively inform the implementation process moving forward.

Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.

We thank the authors of the Letter to the Editor (Kassam et al. 2023) for raising excellent points about our scoping review (Hamza et al. 2023). Our goal with this work was to offer a synthesis and mapping of literary conversations about CBME since 1978 as both a resource and a prompt for further conversations about multiple aspects of competency-based medical education (CBME). The letter authors share that in their own program, ‘those who identified as men felt much better prepared for the transition to CBME than those who identified as women.’ We suggest that gender differences in perception of preparedness are likely more related to persistent findings regarding to confidence gaps between men and women (Ehrlinger and Dunning 2003) than to actual preparation or issues specifically related to CBME. However, the authors do raise an important point about observed risks of gender inequity in assessment. This issue highlights the critically important need for ongoing monitoring for bias in assessment related to gender or other learner characteristics to inform strategies to counteract inequity in all learner assessment, including CBME programs (Klein et al. 2019). We agree with the letter authors’ concerns about the risks of reductionism and implementation issues overshadowing the larger goals of CBME to ensure desired outcomes of training. Attention to assessing and monitoring the progress of all learners, not just those in difficulty, is a core feature of CBME design (van Melle et al. 2019). The letter authors also identify ‘a significant omission regarding the individual and societal impacts of CBME.’ We have analyzed and mapped the societal and individual impact of CBME in the article, with more details included in Supplementary Appendix C (Hamza et al. 2023), and we believe this content draws important focus to the larger goals of CBME to balance with the attention to implementation concerns. We agree with the letter authors about the unfortunate lack of sufficient research into the societal impacts of CBME to date. We propose a challenge to leaders across disciplines to balance vision and strategy with tactics and management as they implement and/or evaluate CBME in their contexts.

BACKGROUND:Competency-based medical education (CBME) received increased attention in the early 2000s by educators, clinicians, and policy makers as a way to address concerns about physician preparedness and patient safety in a rapidly changing healthcare environment. Opinions and perspectives around this shift in medical education vary and, to date, a systematic search and synthesis of the literature has yet to be undertaken. The aim of this scoping review is to present a comprehensive map of the literary conversations surrounding CBME. METHODS:Twelve different databases were searched from database inception up until 29 April 2020. Literary conversations were extracted into the following categories: perceived advantages, perceived disadvantages, challenges/uncertainties/skepticism, and recommendations related to CBME. RESULTS:Of the 5757 identified records, 387 were included in this review. Through thematic analysis, eight themes were identified in the literary conversations about CBME: credibility, application, community influence, learner impact, assessment, educational developments, organizational structures, and societal impacts of CBME. Content analysis supported the development of a heat map that provides a visual illustration of the frequency of these literary conversations over time. CONCLUSIONS:This review serves two purposes for the medical education research community. First, this review acts as a comprehensive historical record of the shifting perceptions of CBME as the construct was introduced and adopted by many groups in the medical education global community over time. Second, this review consolidates the many literary conversations about CBME that followed the initial proposal for this approach. These findings can facilitate understanding of CBME for multiple audiences both within and outside of the medical education research community.

Physician Humanism in CanMEDS 2025 Humanisme des médecins dans CanMEDS 2025 Heather M Waters,1 Anna Oswald,2,3 Evelyn Constantin,4 Brent Thoma,5 J Damon Dagnone6 1Department of Family Medicine, McMaster University, Ontario, Canada; 2Department of Medicine, University of Alberta, Alberta, Canada; 3Royal College of Physicians and Surgeons of Canada, Ontario, Canada; 4Department of Pediatrics, McGill University, Quebec, Canada; 5University of Saskatchewan, Saskatchewan, Canada; 6Department of Emergency Medicine, Queen’s University, Ontario, Canada Correspondence to: Heather M Waters, Associate Professor, Department of Family Medicine, McMaster University; email: watersh@mcmaster.ca Published ahead of issue: Oct 18, 2022; published: Mar 21, 2023. CMEJ 2023, 14(1) Available at https://doi.org/10.36834/cmej.75536 © 2023 Waters, Oswald, Constantin, Thoma, Dagnone; licensee Synergies Partners. This is an Open Journal Systems article distributed under the terms of the Creative Commons Attribution License. (https://creativecommons.org/licenses/by-nc-nd/4.0) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is cited.

We examine whether the effects of research impact on faculty compensation and promotion to full professor differ for male and female associate and full professors in the Faculty of Medicine & Dentistry at the University of Alberta. We exclude faculty with MDs and DDSs and proxy for research impact using the faculty member's h-index, where h represents the number of publications that have been cited at least h times. We find that while the compensation of male faculty members increases by 0.6% for every one-unit increase in the h-index, the compensation of female faculty is essentially uncorrelated with their h-indices. We likewise find that for female faculty to be promoted to full professor they have to have higher research impact proxies than their male peers. Our findings highlight the urgent need for more research on the gendered relationships between research impact and career rewards among faculty.

Purpose Organizational readiness is critical for successful implementation of an innovation. We evaluated program readiness to implement Competence by Design (CBD), a model of Competency-Based Medical Education (CBME), among Canadian postgraduate training programs. Methods A survey of program directors was distributed 1 month prior to CBD implementation in 2019. Questions were informed by the R = MC2 framework of organizational readiness and addressed: program motivation, general capacity for change, and innovation-specific capacity. An overall readiness score was calculated. An ANOVA was conducted to compare overall readiness between disciplines. Results Survey response rate was 42% (n = 79). The mean overall readiness score was 74% (30-98%). There was no difference in scores between disciplines. The majority of respondents agreed that successful implementation of CBD was a priority (74%), and that their leadership (94%) and faculty and residents (87%) were supportive of change. Fewer perceived that CBD was a move in the right direction (58%) and that implementation was a manageable change (53%). Curriculum mapping, competence committees and programmatic assessment activities were completed by >90% of programs, while Conclusion Our study highlights important areas where programs excelled in their preparation for CBD, as well as common challenges that serve as targets for future intervention to improve program readiness for CBD implementation.

Hall and Oswald argue that when clinical competence committees default to progression in the face of inadequate data requires revision to programs of assessment, improved data visualization, and targeted faculty development.

While most case studies consider how programs of assessment may influence residents’ achievement, we engaged in a qualitative, multiple case study to model how resident engagement and performance can reciprocally influence the program of assessment. We conducted virtual focus groups with program leaders from four residency training programs from different disciplines (internal medicine, emergency medicine, neurology, and rheumatology) and institutions. We facilitated discussion with live screen-sharing to (1) improve upon a previously-derived model of programmatic assessment and (2) explore how different resident archetypes (sample profiles) may influence their program of assessment. Participants agreed that differences in resident engagement and performance can influence their programs of assessment in some (mal)adaptive ways. For residents who are disengaged and weakly performing (of which there are a few), significantly more time is spent to make sense of problematic evidence, arrive at a decision, and generate recommendations. Whereas for residents who are engaged and performing strongly (the vast majority), significantly less effort is thought to be spent on discussion and formalized recommendations. These findings motivate us to fulfill the potential of programmatic assessment by more intentionally and strategically challenging those who are engaged and strongly performing, and by anticipating ways that weakly performing residents may strain existing processes.

PURPOSE:This study evaluated the fidelity of competence committee (CC) implementation in Canadian postgraduate specialist training programs during the transition to competency-based medical education (CBME).METHODS:A national survey of CC chairs was distributed to all CBME training programs in November 2019. Survey questions were derived from guiding documents published by the Royal College of Physicians and Surgeons of Canada reflecting intended processes and design.RESULTS:Response rate was 39% (113/293) with representation from all eligible disciplines. Committee size ranged from 3 to 20 members, 42% of programs included external members, and 20% included a resident representative. Most programs (72%) reported that a primary review and synthesis of resident assessment data occurs prior to the meeting, with some data reviewed collectively during meetings. When determining entrustable professional activity (EPA) achievement, most programs followed the national specialty guidelines closely with some exceptions (53%). Documented concerns about professionalism, EPA narrative comments, and EPA entrustment scores were most highly weighted when determining resident progress decisions.CONCLUSIONS:Heterogeneity in CC implementation likely reflects local adaptations, but may also explain some of the variable challenges faced by programs during the transition to CBME. Our results offer educational leaders important fidelity data that can help inform the larger evaluation and transformation of CBME.