Programs worldwide are introducing programmatic assessment and Entrustable Professional Activities (EPAs) with strong theoretical foundations and growing research evidence regarding potential benefits. In Australia, the new National Framework for Prevocational Medical Training includes four EPAs for doctors in PGY1-2, who rotate through different specialties to gain broad experience: EPA1:Clinical assessment, EPA2:Recognition and Care of the Acutely Unwell Patient, EPA3:Prescribing, EPA4:Team communication. Little is known about how this new education model will work in practice. Hence a pilot study was conducted to (1) Investigate how feasibility, acceptability, educational value and overall utility of EPAs can be maximised, and (2) Clarify challenges and enablers of EPA implementation, operationalised via EPA assessments. The study was conducted during intern terms 2–4, 2024 (10 weeks each) in a metropolitan health service. EPA assessments (EPAA) were implemented for interns working in eleven medical units across four specialties (medicine, surgery, mental health, emergency). Change strategies were designed using Kotter’s 8-step and COM-B models. Action-research methodology was used and each 5-week cycle involved: enacting plans, gathering user data, analysing data, and co-designing improvements with expert and stakeholder panels. Quantitative and qualitative user data were collected from all EPAA e-forms, plus surveys and focus groups conducted each cycle. In total, 738 EPAAs were submitted by 119 interns and 283 assessors. The frequency order was: EPA1(highest)-EPA4-EPA3-EPA2. The majority took 10 min or less to complete, and most were assessed by registrars, except in Emergency where consultants predominated. EPAA activity varied substantially across individual units, even within one specialty. User perspectives indicated that EPAAs' educational value depended primarily on the quality of the feedback discussion. Key challenges identified were establishing a trained assessor cohort, fitting EPAAs into workflow, refining the form design and rating scale (National Framework scale used in Phase 1 and modified Ottawa-scale in Phases 2–3). Multiple revisions to implementation were co-designed and executed across the pilot. Practical strategies to support implementation were clarified. This large pilot provides valuable insights into EPA implementation according to the National Framework. Clarifying challenges and enablers may assist others. Optimising EPAA quantity, rating scale, form and feedback quality is important due to impacts on realising educational value and providing robust performance data for progress decisions.

While competency-based programmatic assessment systems have become widespread worldwide, there are few comparisons of competence assessment in orthopaedic surgery between countries. In this paper, we provide a narrative overview of current systems in the United States, Canada, United Kingdom and Ireland, and Australia. We explore what types of workplace-based assessments are used, how many data points are gathered to inform entrustment and progression decisions and whether there is validity evidence using a modified Kane framework. Significant concordance and some variation were found between the programs, and the issues identified are discussed along with the challenges and potential solutions for enhancing competence assessment.

Introduction: Competency-Based Health Professions Education (CBHPE) is an outcomes-based approach to planning and conceptualizing the training of health professionals to serve population needs. Despite its uptake, ongoing debate persists regarding the assumptions underpinning CBHPE, its intended purposes, and how success should be defined and evaluated. This contributes to variable implementation and difficulty aligning core features with meaningful outcomes. To date, no published theory of change has articulated how CBHPE’s core components provide a framework for later evaluation of intended outcomes.Methods: Using Funnell and Rogers’ purposeful program theory, the authors, all members of the International Competency-based Education Health Professions Educators Collaborative (ICBHPE), developed a model theory of change. The authors conducted a broad review of CBHPE literature, supplemented by CBE frameworks from adjacent educational contexts to strengthen underdeveloped constructs in implementation and evaluation. Findings were iteratively synthesized into a program theory and refined through repeated cycles of large-group feedback, consensus building, and member checking within ICBHPE.Results: The authors developed a CBHPE Outcomes Theory of Change that articulates key assumptions, mechanisms of impact, and expected outcomes associated with CBHPE implementation. The framework specifies how van Melle’s CBHPE core components are theorized to function across phases of implementation and identifies proximal and distal outcomes that can reasonably be expected, based on intentional, deliberate program evaluation at each implementation and evaluation stage.Discussion: The CBHPE Outcomes Theory of Change offers scholars and implementers a practical guide to enhance the quality and fidelity of implementation, evaluation, and research efforts related to CBHPE worldwide. Using this guide as a framework, programs can plan, implement, and then examine whether the intended outcomes of CBHPE have been realized.

The transition from training to independent practice in Emergency Medicine is challenging. Early career physicians need specific knowledge to function in new roles. However, this knowledge can be unwritten and implicit. It is unclear what skills newly graduated emergency physicians need to succeed in their early independent practice. We wished to understand these elements in order to inform further development of the current transition to practice curriculum. We interviewed 23 emergency physicians in their first 5 years of practice at an urban academic center to explore the knowledge they acquired in early practice. Employing a constructivist stance, data collection and analysis occurred iteratively until thematic saturation. We constructed themes using inductive thematic analysis. Newly graduated emergency physicians identified the knowledge needed in early clinical practice. This knowledge was unwritten, dynamic, nuanced, difficult to transfer, and vital for them to function effectively. Knowledge was needed in four facets of clinical practice: patient interaction, learner supervision, institutional processes and group culture. Within patient interaction, they required skills and knowledge for their responsibility, gestalt, decision-making and communication. They needed skills to supervise learners, delegate them with tasks and provide feedback. To function effectively, they needed to learn institutional processes and how to manage departmental tasks. They also required knowledge of tacit group norms about work efficiency, code of conduct and patient care. Newly graduated physicians sought strategies to manage aspects within each of these four facets. Specific knowledge is necessary in multiple facets of independent Emergency Medicine practice. Such knowledge is often unwritten and contextual, but the associated skills and strategies can be made explicit. Educators should incorporate these components in their transition-to-practice curriculum to support graduating trainees in becoming successful attending physicians.

The peer review process supports authors by providing feedback on manuscripts from external, expert readers. However, not all reviews are supportive. Some reviews are just painful. But just how painful are they? In this study, we set out to validate a matrix describing the levels of pain authors experience in response to the sting of peer reviewer comments. The study was carried out in two phases. In Phase 1, we developed a matrix combining two scales—i.e. the Suffering Scale and the Grind Gauge. The first categorises review‐induced pain across four levels, with Level 1 being the least pain and Level 4 the greatest pain. The second categorises the amount of work required by the author to respond to and address reviewer comments, with Level 1 being the least amount and Level 4 the greatest amount. In Phase 2, we tested the performance of the matrix by recruiting multiple, global study sites to provide performance data. This work resulted in the development and validation of the Reviewer‐Imposed Pain (RIP) matrix. A statistician analysed our data and assures us that the RIP matrix is now a validated tool. Our study shows that the pain associated with academic peer review affects physiologic, affective, and cognitive dimensions. This tongue‐in‐cheek paper pokes fun at the peer review process; however, the response from survey participants suggests that the process is not necessarily funny. Peer review is essential for advancing science; however, for these advancements to occur, peer reviewer comments need to be constructive. The RIP matrix encourages both authors and reviewers to reflect on the impact of reviewer comments. This is essential because, as previous research has illustrated and as Voltaire succinctly stated, pain is real.

Introduction:Clinical handover (or handoff), the transfer of patient care between providers, is essential in Emergency Medicine. Poor communication during handover can threaten patient safety. Existing literature views handover as an information transfer; little research has examined the human interaction within handover and how it affects patient safety. We sought a more nuanced understanding of handover to improve this critical aspect of Emergency Medicine care. Methods:We used constructivist grounded theory to explore handover as a social phenomenon. We invited staff emergency physicians in an academic, tertiary care hospital to participate in semi-structured interviews, which were recorded, de-identified, and transcribed. The research team analyzed the transcripts in multiple progressively interpretive analytical stages: initial, focused, and theoretical. Cultural-Historical Activity Theory (CHAT) provided a framework for our data analysis. CHAT situates an individual's action into an activity system in which components interact, producing tensions. We sought to attain theoretical sufficiency by revising the interview guide and recruiting participants for specific insights. Results:We interviewed sixteen (16) participants. During handover, the object was for the incoming physician (IP) to accept the handover plan from the outgoing physician (OP). We identified a rule requiring minimal handover plans that were realistic and dichotomous. The divisions of labor were tasks. During the interaction, emotional reactions such as judgment or compassion, and behavioral reactions such as pushback or acceptance could have resulted. There were unintended consequences from the handover process that impacted members' self-worth and generated threats to patient safety. Conclusions:We built a novel and more nuanced framework of handover in the emergency department that involves multiple interactions and tensions. Using this new model, emergency departments should adjust their handover processes to enhance group dynamics and patient safety.

Assessor stringency and leniency (ASL)-an assessor's tendency to award low or high scores-has a significant effect on workplace-based assessments. Outliers on this spectrum have a disproportionate effect. However, no method has been published for quantifying ASL or identifying outlier stringent or lenient assessors using workplace-based assessment data. The authors propose the mean delta method, which compares the scores that an assessor awards to trainees with those trainees' mean scores. This novel, simple method can be used to quantify ASL and identify outlier assessors without requiring specialized statistical knowledge or software. As a worked example, the mean delta method was applied to a set of end-of-shift assessments completed in a large Canadian academic emergency department from July 1, 2017, to May 31, 2018, and used to examine the net effect of ASL on learners' assessment scores. A total of 3,908 assessments were completed by 99 assessors for 151 trainees, with a median (interquartile range) of 37 (12-39) completed assessments per trainee. Using cutoff values of 1.5 and 2 standard deviations, a total of 11 and 3 outlier assessors were identified, respectively. Moreover, ASL changed overall scores by more than the mean difference between years of training for nearly 1 in 4 learners. The mean delta method was able to quantify ASL and identify outlier lenient and stringent assessors. It was also used to quantify the net effect of ASL on individual trainees. This method could be used to further study outlier assessors, to identify assessors who may benefit most from targeted coaching and feedback, and to measure changes in assessors' tendencies over time or with specific intervention.

Introduction:The field of medical education (ME) has grown substantially over the past decades, yet questions remain about its scope and boundaries. This study examines how research topics and institutional collaborations have evolved in ME from 2000 to 2019. Methods:Adopting a post-positivist stance and using bibliometric network analyses, we examined metadata from 31,338 publications across 22 core ME journals indexed in the Web of Science. We analyzed trends in institutional collaboration and the development of research themes. Extracted metadata included authors' institutional affiliations and KeyWords Plus (n = 18,218). Bibliometric analyses were conducted using VOSviewer, a widely used tool for network mapping. We generated co-authorship networks to trace institutional collaboration and co-word networks to identify thematic clusters. Results:Co-authorship networks revealed increasing collaboration, with U.S. institutions remaining central and Canadian and Dutch institutions gaining prominence. Co-word analyses identified three stable clusters-teaching and learning, quantitative, and psychosocial-with teaching and learning dominant across all periods and the quantitative cluster expanding in recent years. Discussion:Findings show the consolidation of teaching and learning as the foundation of ME, alongside diversification through quantitative and psychosocial themes. Growing collaborations suggest the field's maturation, though geographic imbalances persist. Limitations include reliance on a restricted set of Web of Science journals, which overrepresent English-language and highly cited publications, and the use of KeyWords Plus as a proxy for themes. This study offers an evidence-based mapping of ME's evolution and provides a framework for future research on the interdisciplinary and global dynamics of the field.

Introduction:Assessor stringency/leniency (ASL), the tendency for an assessor to consistently provide low or high scores, has been shown to have educationally important effects on learner assessment scores in multiple settings. To date, there is no clear understanding of the underlying drivers of ASL in the context of medical education. Some authors have hypothesized a link between personality traits and ASL, but there is currently insufficient data to reach any conclusions. This study seeks to determine whether there is a significant association between physician assessors' personality traits and ASL. Methods:This prospective cohort study was conducted at an academic emergency department in Ottawa, Canada. Participating assessors volunteered to complete the IPIP-Neo 120, a personality questionnaire based on the five-factor model. All end-of-shift assessments completed between July 1, 2021, and June 30, 2022, were collected, and ASL was quantified for each assessor using the mean delta method. Linear regression was used to assess the correlation between personality scores and ASL. Results:A total of 2127 assessments, representing 184 learners, were analyzed. Twenty-five assessors were enrolled, with a wide distribution of assessor personality scores for each trait. While there was a trend toward leniency with increasing assessor extraversion, this did not reach statistical significance (p = 0.07, R 2 = 0.13). There was no significant link between other personality traits and ASL. Conclusion:Integrating our findings with the existing literature, we conclude that personality traits are likely not educationally important drivers of ASL in medicine. Future research should examine other possible contributors to ASL in medical education.

Bibliometric network analysis is an analytical approach that enables researchers to visualize the relationships between a set of research items (e.g., journal articles, books). There are 3 types of bibliometric network analyses, and multiple tools to conduct the analysis and visualize results (e.g., VOSviewer , 1Gephi2 ). For health professions educators, bibliometric network analysis is valuable for discovering the field's emerging trends, popular topics, and multidisciplinary relationships. 3,4.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

In the past decade, the Canadian system of postgraduate medical education has been transformed with the implementation of a new approach to competency based medical education called Competence by Design. The Royal College of Physicians and Surgeons of Canada (Royal College) developed an approach to time-variable competency based medical education and adapted that design for medical, surgical, and diagnostic disciplines. New educational standards and entrustable professional activities consistent with this approach were co-created with 67 specialties and subspecialties, and implementation was scaled up across 17 universities and over 1000 postgraduate training programs. Partner engagement, systematic design of workshops to create discipline specific competency-based standards of education, and agile adaptation were all key ingredients for success. This paper describes the strategies applied by the Royal College, lessons learned regarding transformative change in the complex system of postgraduate medical education, and the current status of the Competence by Design initiative. The approach taken and lessons learned by the Royal College may be useful for other educators who are planning a transformation to CBME or any other major educational reform.

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Competency based medical education is being adopted around the world. Accreditation plays a vital role as an enabler in the adoption and implementation of competency based medical education, but little has been published about how the design of an accreditation system facilitates this transformation. The Canadian postgraduate medical education environment has recently transitioned to an outcomes-based accreditation system in parallel with the adoption of competency based medical education. Using the Canadian example, we characterize four features of an accreditation system that can facilitate the implementation of competency based medical education: theoretical underpinning, quality focus, accreditation standards, and accreditation processes. Alignment of the underlying educational theories within the accreditation system and educational paradigm drives change in a consistent and desired direction. An accreditation system that prioritizes quality improvement over quality assurance promotes educational system development and progressive change. Accreditation standards that achieve the difficult balance of being sufficiently detailed yet flexible foster a high fidelity of implementation without stifling innovation. Finally, accreditation processes that recognize the change process, encourage program development, and are not overly punitive all enable the implementation of competency based medical education. We also discuss the ways in which accreditation can simultaneously hinder the implementation of this approach. As education bodies adopt competency based medical education, particular attention should be paid to the role that accreditation plays in successful implementation.

Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Competency based medical education (CBME) is a global movement to reform health professions education (HPE).In contrast to the traditional time-based training, it is an educational design approach that emphasizes necessary learner abilities, greater learner-centeredness, and better alignment with public's needs for healthcare providers.Unlike numerous previous HPE innovations, CBME is framed as a complex transformational change with multiple proposed elements [1].Training programs employing a competency-based approach can have five core components to operationalize their implementation with fidelity: outcome competencies, progressive sequencing, tailored learning experiences, competency-focused instruction, and programmatic assessment [2].Health professions education institutions, disciplines, and programs across the globe have been implementing CBME at an ever-increasing rate.The CBME movement began with early innovations such as the CanMEDS framework [3-7], the ACGME Outcomes Project [8-10], entrustable professional activities [11,12], the Dutch CBME curriculum [13,14], the Cincinnati observable practice activities [15], the College of Family Physicians of Canada Triple C Curriculum [15-17], and the University of Toronto Orthopedic program [18].CBME has now spread worldwide with implementations in, for example, Switzerland, Taiwan, Australian Orthopedic training [19], Finland, South Africa, Brazil, and many more countries.Competency based education is now a part of many health professions beyond medicine, including nursing [20] and veterinary medicine [21].Despite the global implementation effort, the current discourse around CBME is still largely focused on theoretical underpinnings and potential [22,23] and countered with skeptical criticisms [24,25].There are early efforts to evaluate and clarify the outcomes of these CBME curricula [26,27].However, what is needed at this stage of the diffusion of these innovations is sharing CBME praxis: examples of the application of the CBME approach to real-life HPE systems and the lessons learned for others who follow.In this special issue of Perspectives on Medical Education, a team of educators has come together to describe a unique Canadian configuration of CBME implementation called Competence by Design (CBD).Worldwide, the majority of CBME innovations have focused on implementing

Coaching is an increasingly popular means to provide individualized, learner-centered, developmental guidance to trainees in competency based medical education (CBME) curricula. Aligned with CBME's core components, coaching can assist in leveraging the full potential of this educational approach. With its focus on growth and improvement, coaching helps trainees develop clinical acumen and self-regulated learning skills. Developing a shared mental model for coaching in the medical education context is crucial to facilitate integration and subsequent evaluation of success. This paper describes the Royal College of Physicians and Surgeons of Canada's coaching model, one that is theory based, evidence informed, principle driven and iteratively and developed by a multidisciplinary team. The coaching model was specifically designed, fit for purpose to the postgraduate medical education (PGME) context and implemented as part of Competence by Design (CBD), a new competency based PGME program. This coaching model differentiates two coaching roles, which reflect different contexts in which postgraduate trainees learn and develop skills. Both roles are supported by the RX-OCR process: developing Relationship/Rapport, setting eXpectations, Observing, a Coaching conversation, and Recording/Reflecting. The CBD Coaching Model and its associated RX-OCR faculty development tool support the implementation of coaching in CBME. Coaching in the moment and coaching over time offer important mechanisms by which CBD brings value to trainees. For sustained change to occur and for learners and coaches to experience the model's intended benefits, ongoing professional development efforts are needed. Early post implementation reflections and lessons learned are provided.

Transformative changes in health professions education need to incorporate effective faculty development, but few very large-scale faculty development designs have been described. The Royal College of Physicians and Surgeons of Canada's Competence by Design project was launched to transform the delivery of postgraduate medical education in Canada using a competency-based model. In this paper we outline the goals, principles, and rationale of the Royal College's national strategy for faculty and resident development initiatives to support the implementation of Competence by Design. We describe the activities and resources for both faculty and trainees that facilitated the redesign of training programs for each specialty and subspecialty at the national level, as well as supporting the implementation of the redesign at the local level. This undertaking was not without its challenges: we thus reflect on those challenges, enablers, and the lessons learned, and discuss a continuous quality improvement approach that was taken to iteratively inform the implementation process moving forward.