OBJECTIVE:The objective was to combine a set of linguistic analyses for efficiently analyzing large amounts of qualitative data from student evaluations of experiential sites and preceptors for quality assurance, evaluate their accuracy in coding these evaluations compared with manual processes, and assess the correlation between quantitative and qualitative sections of student evaluations. METHODS:A Python script was written to analyze comments from the qualitative sections of deidentified student evaluations of their preceptor and site. Each comment was analyzed using lexical, sentiment, and semantic analyses and given an aggregate rating, ranging from very negative to very positive. The script was iteratively refined using randomly selected samples of evaluations, subsequently tested to verify accuracy, and then used to analyze a cohort of evaluations. The correlation between the quantitative and qualitative sections was assessed to determine the strength of the relationship between the 2 sections. RESULTS:After refinements to the script, 93% of the qualitative sections of evaluations showed the correct valence (eg, positive, neutral, negative), as measured against agreed-upon experiential faculty ratings. Of the 7% coded incorrectly, 2% were programmatically relevant. The correlation between quantitative and qualitative sections was weak but significant. CONCLUSION:The linguistic analyses were able to identify sites potentially requiring further review, holding promise for alleviating burden on experiential staff by limiting the quantity of qualitative data to manually review. Correlation results suggested that quantitative data were not sufficient to capture potential concerns. Further research is necessary to evaluate integration into real-time experiential education workflows.

ABSTRACT Misconceptions in pharmacology can undermine learning and compromise both clinical and scientific reasoning, yet few validated tools exist to identify them. Consequently, we developed and validated the Pharmacology Concept Inventory (PCI), which can be used to identify misconceptions, assess learning gains, and evaluate teaching effectiveness. This PCI was designed based on the IUPHAR‐Education Section (IUPHAR‐Ed) Core Concepts of Pharmacology Project, addressing eight core concepts: drug efficacy, drug‐target interaction, steady‐state concentration, structure–activity relationship, drug tolerance, drug bioavailability, volume of distribution, and drug clearance. A triangulated design strategy integrated theoretical frameworks, expert review, and student perspectives. Experts examined quality, content validity, and cognitive alignment. The pilot PCI was then administered to a student cohort to evaluate its psychometric properties, providing preliminary evidence for further refinement. Item‐level content validity indices ranged from 0.67 to 1.00, with a scale‐level average of 0.93. Seventy students completed the pilot survey, leading to the exclusion of items with low discrimination and reliability. Items on drug‐target interaction were removed due to consistently poor performance. The final PCI included 26 items covering seven concepts, with strong discrimination indices (0.36–0.75) and difficulty indices (0.26–0.71). Internal consistency was high (Cronbach's alpha = 0.91), and concept‐level reliability ranged from 0.64 to 0.85. The PCI provides strong evidence for identifying misconceptions and assessing learning outcomes through a pre–post‐test approach. Although the PCI currently addresses only a subset of concepts, continued refinements informed by surveys and interviews will enhance its utility and expand its scope for concept‐based learning and curriculum evaluation.

Comprehensive exams are a standard component of doctoral-level training programs in epidemiology. In this manuscript, we describe current practices in the administration of comprehensive examinations across doctoral programs in epidemiology and discuss variation in the content, format, and structure of these exams. We present results from a survey about the comprehensive exam process at thirty-three different epidemiology programs. We also discuss a symposium focused on comprehensive exams that was part of the 2025 Society for Epidemiologic Research annual meeting. The symposium included speakers with specific expertise on designing comprehensive exams as well as faculty from several different epidemiology programs across North America.. The survey found important differences in format (ie, written, oral, general epidemiology, substantive topics), when the exams are administered, and what level of epidemiology knowledge they are targeting. In this manuscript, we also discuss common challenges programs face, including creating and grading the exams. By sharing this information with a wide audience of epidemiologists, we aim to facilitate discussion to improve the comprehensive exam process for doctoral students in our field. This work also fits with abroader goal to initiate discussion about doctoral-level training in epidemiology.

Background Generative AI (GenAI) has increasingly been used in ways to support health professions education but the utility of it to support research skill development for pharmacy residents is not well established. Objectives We examined perceptions of GenAI for supporting the development of research and evaluation skills among dual master’s/resident students in a health system administration and leadership degree program. Methods This was a convergent parallel mixed-methods design study situated within a two-year dual masters/residency Master in Pharmaceutical Sciences program with a specialization in Health System Pharmacy Administration. Students completed Part 1 and Part 2 of an activity involving GenAI feedback on research design and were asked to complete a survey involving Likert-type questions and open-ended questions. Results Nearly all students (n=46) completed Part 1 (96% response rate) and Part 2 (n=46, 100% response rate). Qualitative analysis of responses from Parts 1 and 2 produced four themes that aligned the quantitative findings from Part 1: helpful and informative feedback, skepticism around GenAI feedback, reflection on work and course learnings, and desire to learn more and use GenAI in the future. Conclusion Residency students found GenAI to be supportive for developing research plans situated within their healthcare institutions and in future contexts. But they also expressed a need for critical review of GenAI’s suggestions. GenAI presents a possible avenue for support in research skill development with appropriate guidance and mentorship.

OBJECTIVE:To prospectively compare the psychometric performance of 3-option versus 4-option multiple-choice question (MCQ) examinations and evaluate differences in item difficulty, item discrimination, examination reliability, examination performance, and elapsed testing time. METHODS:This randomized crossover study was conducted in 2 required Doctor of Pharmacy courses at a single school of pharmacy, including a first-year pharmacokinetics course and a third-year pharmacotherapy course. Students were assigned to alternating sequences of 3-option and 4-option MCQ examinations throughout the study period. Four-option examinations were developed first, after which 1 distractor was removed from each item to create the corresponding 3-option version. Item-level psychometric measures, examination-level outcomes, and elapsed testing time were compared between formats using paired analyses and linear mixed-effects modeling. RESULTS:Three-option MCQs were associated with students answering approximately 1 additional question correctly on a 50-item MCQ test compared with 4-option MCQs. No differences were observed in point-biserial discrimination or Kuder-Richardson 20 reliability estimates between examination formats. Course-specific analyses demonstrated differences in item difficulty within pharmacokinetics examinations but not pharmacotherapy examinations. Examinations containing 4-option MCQs also required longer testing times for students. CONCLUSION:Reducing MCQs from 4 to 3 answer options did not meaningfully compromise examination reliability or item discrimination in 2 required pharmacy courses. Although 4-option MCQs produced slightly greater difficulty and longer testing times, the practical educational benefit of routinely requiring a fourth answer option may be limited when distractors are not consistently plausible or functioning effectively.

OBJECTIVE:To compare cumulative final examination performance between mastery learning (ML) and team-based learning (TBL) and conduct an exploratory characterization of differences in instructional complexity between the two models. METHODS:This retrospective study compared seven years of student performance data from a TBL course format (2012-2018, n=996) with seven years following implementation of ML (2019-2025, n=1,078). Course content, sequencing, learning objectives, and cumulative examination blueprint remained relatively stable across years. The primary outcome was student performance on a cumulative final examination. Final examination scores were compared between instructional models using Welch's independent samples t-test. Effect sizes were calculated using Cohen's d. An exploratory retrospective analysis used instructor-informed ratings to characterize relative instructional complexity between formats. RESULTS:Students in the ML format achieved statistically significantly higher examination scores compared with students in the TBL format (ML=90.0%, TBL=88.0%, mean difference=2.0 percentage points, p<.001, d=0.24). Results remained consistent when COVID-affected cohorts were excluded from analysis. ML was also associated with lower score variability (SD=7.9 vs 8.9), suggesting greater consistency in student performance. The exploratory instructor-informed complexity analysis suggested that TBL concentrated relative instructional complexity within synchronous classroom coordination and team management, whereas ML concentrated complexity within assessment infrastructure, reassessment processes, and individualized learner support. CONCLUSION:Both TBL and ML were associated with high cumulative final examination performance despite substantial differences in instructional organization and operational demands. Selection of an instructional model should depend on alignment among learning goals, institutional resources, and implementation context rather than small differences in examination outcomes.

OBJECTIVE:To develop a set of research entrustable professional activities (EPAs) for practicing pharmacists to support research training and assessment. METHODS:A mixed-methods, multiphase study was conducted. In Phase 1, pharmacy research experts participated in a focus group to generate a preliminary list of research tasks using a structured nominal group technique. Tasks were organized into domains representing stages of the research process. In Phase 2, the research team refined the tasks into draft EPAs and evaluated them using a modified EQual rubric assessing structural quality across 3 domains. EPAs were revised based on quantitative scores and qualitative feedback. In Phase 3, an external panel of pharmacy research experts evaluated the refined EPAs using an e-Delphi process. Prespecified cutpoints were applied across phases to guide progression and final selection. RESULTS:Thirty-five pharmacy research experts participated in the Phase 1 focus group, generating 34 research tasks across 7 domains. In Phase 2, 5 research team members evaluated and refined these tasks, resulting in 22 draft EPAs. In Phase 3, 33 external experts evaluated the draft EPAs; 2 did not meet prespecified cutpoints and were removed. The final framework included 20 research EPAs with mean evaluation scores ranging from 4.03 to 4.64. CONCLUSION:This study presents the first set of research EPAs developed for pharmacy practice. These EPAs define observable research activities that pharmacists may be entrusted to perform and provide a structured, adaptable framework to support curricular design, assessment, and progression of research competency in pharmacy education.

Artificial intelligence (AI), including generative artificial intelligence (GenAI), is increasingly being incorporated into health care practice and academic environments, creating an urgent need for pharmacy education programs to prepare learners to engage with these tools responsibly. Accrediting bodies and professional organizations emphasize innovation, digital literacy, and readiness for contemporary practice; however, specific guidance on how GenAI should be integrated into pharmacy education remains limited. As a result, pharmacy educators face uncertainty related to pedagogical alignment, ethical use, assessment integrity, and student reliance on AI-generated outputs. The purpose of this "how-to" guide is to assist pharmacy educators and training program leaders with practical strategies and examples for integrating GenAI into teaching and assessment across pharmacy education. This guide presents foundational principles to support responsible GenAI use, followed by a step-by-step framework that addresses identification of instructional needs, selection of appropriate GenAI modalities, activity design, student preparation for critical AI use, and assessment and refinement of AI-enabled learning activities. Common instructional contexts and applications are illustrated using real-world examples, including clinical reasoning exercises, communication skill development, scalable assessment, scholarly writing support, and formative feedback. Key challenges encountered during GenAI integration are synthesized, including overreliance on AI, inaccurate or biased outputs, variability in AI performance, and workflow considerations for faculty and learners. Specific mitigation strategies and design decisions are provided to support intentional implementation while maintaining academic rigor and professional standards. By focusing on instructional strategies rather than specific tools, this guide offers adaptable recommendations to support pharmacy educators in leveraging GenAI to enhance learning and prepare trainees for AI-enabled pharmacy practice.

Purpose This study illustrates the use of design thinking (DT) as a structured, participatory approach to explore contemporary complex challenges in health professions admissions, including the rise of generative AI, remote interviews, and the elimination of standardized admissions tests. This work focuses on stakeholder-driven problem framing and idea generation rather than evaluating outcomes. Methods A two-hour workshop engaged 15 purposively sampled stakeholders, including faculty, staff, application readers, and student ambassadors, in collaborative problem framing, ideation, and rapid prototyping of admissions concepts. Generative artifacts (brainstorming outputs, reflection worksheets, facilitator notes) and post-session surveys were analyzed using thematic synthesis and descriptive statistics to characterize emergent ideas and participant perspectives. Major findings Participants generated ideas that clustered into three major themes: interview restructuring, GenAI integration and compliance, and broadening of admissions criteria, illustrating how stakeholders reframed challenges and proposed diverse solution pathways. Survey responses reflected descriptive indicators of participant experience, suggesting the workshop supported creative problem solving (Mean 4.7 ± 0.5), idea generation (4.8 ± 0.4), and collaboration (4.4 ± 0.5). Conclusions Findings suggest that DT offers a structured, iterative framework for stakeholder-driven idea generation and problem reframing in an evolving admissions context. The workshop demonstrates the potential of collaborative, reflective processes to surface assumptions and generate diverse perspectives to inform future exploration of admissions practices.

Purpose Providing effective peer feedback plays an important role in collaborative learning and is an essential professional skill for students to develop. This study investigates whether generative AI can improve the quality of peer feedback, and whether it influences students’ perceptions of the feedback received. Methods An experimental design was employed involving 129 third-year Doctor of Pharmacy (PharmD) students. Participants were randomized into two groups: self-generated (SG) feedback and AI-assisted (AI) feedback. The SG group provided feedback independently (i.e., the feedback was crafted on their own, without the use of an AI prompt) using the task, gap, action (TGAP) framework. The AI group utilized generative AI to create feedback based on a given prompt that aligned with the TGAP framework. The prompt was partially completed, requiring students to add behaviors, professional skills (e.g., communication, interpersonal skills), or knowledge related to each student they were evaluating. Feedback was coded and analyzed, and students’ perceptions were measured using the Feedback Perceptions Questionnaire. Results A total of 353 peer feedback comments were analyzed (162 self-generated, 191 AI-assisted). The AI-assisted group achieved significantly higher median scores across all TGAP criteria (Task: H(1) = 27.32, p < 0.001; Gap: H(1) = 89.32, p < 0.001; Action: H(1) = 86.86, p < 0.001). Specifically, 37.0% of students in the SG group compared to 61.3% of students in the AI group provided specific feedback on what their peers did well. For areas of improvement, 13.6% of students in the SG group provided specific areas to improve upon compared to 55.0% of students in the AI group. Only 22.8% of students in the SG provided feedback on how their peers could improve moving forward, compared to 72.8% in AI group. Student perceptions of feedback were positive, with only 3% of the AI group reporting negative feelings about the feedback they received compared to 12.7% in the SG group; this difference was not statistically significant (χ²(1) = 2.97, p = 0.085). Conclusions The study demonstrates how the strategic use of AI can be used to improve the quality of peer feedback. Future research should explore the long-term effects of AI-assisted feedback on students’ independent feedback skills and investigate behavioral changes resulting from improved feedback quality.

OBJECTIVE:The purpose of this study was to evaluate preliminary results of a centralized online referral system that facilitated expedient intervention in experiential environments. METHODS:An online referral system was established for students, faculty, staff, and preceptors to bring immediate student concerns to the school's attention and request real-time support. Experiential academic and personal concerns submitted by preceptors were managed collaboratively by student affairs and experiential education personnel. Referrals from 2018 to 2022 were deidentified and analyzed using qualitative methods. Quantitative and qualitative data from the 2019 and 2023 American Association of Colleges of Pharmacy Preceptor Surveys related to student support and the referral system were assessed using descriptive statistics and thematic analysis to determine preceptor satisfaction with the referral system. RESULTS:Fifty-five referrals were submitted, 46 of which involved students in the second professional year. A total of 182 codes were applied to the data. The most common codes were related to Professionalism (n = 94, 51.6%), Personal Issues (n = 50, 27.4%), and Academic Concerns (n = 28, 15.3%). Preceptors agreed or strongly agreed that they knew how to utilize processes for academic misconduct (86.3%), professional misconduct (90.4%), and harassment/discrimination (88.1%), and survey responses ranked higher compared to national and peer institution benchmarks. Preceptors positively endorsed the referral system and support from the school. CONCLUSION:This study found similar preceptor concerns compared to previous studies. Preceptors indicated knowledge of support systems and positively regarded the system and school support. These initial results suggest the system is a promising method to support student success. More studies are needed on interventions applied and student success outcomes.

Microcredentials are an emergent tool to support knowledge and skill development. Despite their growing popularity in medical education – and higher education more broadly – it is unclear how these strategies have been utilized to support continuing professional development in the health professions. A rapid systematic review was conducted to explore the current relevant literature due to the timely and evolving nature of microcredentials. PubMed, Embase, and ERIC were used for the article search. Of the 290 relevant articles found from the searches, a total of 11 articles were included after abstract and full-text screenings. All articles used in this review were published within the past 10 years. Microcredentials were used across various professions, covered a wide range of topics, and employed various teaching strategies. The definitions used for key terms like microcredential were inconsistent across articles.

OBJECTIVE:This study aimed to use the Consolidated Framework for Implementation Research to identify key determinants that impact the successful integration of cultural intelligence training in Doctor of Pharmacy classes and develop recommendations to address the barriers to such training. METHODS:Terms related to cultural intelligence were searched in PubMed, Embase, CINAHL, Scopus, ProQuest Dissertations and Theses, ERIC, and PsycInfo. Articles were imported into Covidence and screened for content related to cultural intelligence in Doctor of Pharmacy programs, specifically in classroom settings. Forty-eight articles were reviewed using deductive coding with Consolidated Framework for Implementation Research determinants. RESULTS:The literature was highly descriptive of the design aspect of the Innovation Domain; the intended advantages of specific course design and content were presented in detail. The Outer Setting Domain was represented by the Accreditation Council for Pharmacy Education standards and the Inner Setting Domain was represented through pharmacy school mission statements. Reflection and evaluation were the focal points of many articles because these were used as measures of student learning and sources of feedback for novel training. CONCLUSION:Schools of pharmacy use various strategies to implement cultural intelligence trainings, and key factors include reflections, local interests, and regional demographics. The strategy of developing and implementing a specific tool for quality monitoring may aid in prioritizing these interests in a more intentional manner while providing students with a clear reference for their learning experiences.

Purpose Providing and receiving peer feedback is an essential professional skill and plays an important role in collaborative learning. However, providing effective feedback is challenging and time-consuming. This study investigates whether generative AI can improve the quality of peer feedback and influence students' perceptions of the feedback received. Methods An experimental design was employed involving 129 third-year Doctor of Pharmacy (PharmD) students. Participants were randomized into two groups: self-generated (SG) feedback and AI-assisted (AI) feedback. The AI group utilized generative AI to create feedback based on a given prompt. The prompt was partially completed, requiring students to add particular behaviors or professional skills related to each student they were evaluating. The SG group provided feedback independently (i.e., the feedback was crafted on their own, without the use of an AI prompt). Feedback was coded and analyzed using the task, gap, action (TGAP) framework, and students' perceptions were measured using the Feedback Perceptions Questionnaire (FPQ). Results The AI-assisted feedback group produced significantly higher quality comments across each of the three feedback criteria. Specifically, the AI group provided more detailed and specific feedback on peers' strengths, areas for improvement, and actionable suggestions for future performance. Students in the AI group received the feedback positively. Conclusions The study demonstrates how the strategic use of AI can be used to improve the quality of peer feedback. Future research should explore the long-term effects of AI-assisted feedback on students' independent feedback skills and investigate behavioral changes resulting from improved feedback quality.

Research and evaluation skills are essential in healthcare education. Instructors frequently employ collaborative learning models to teach these competencies; however, delivering timely and personalized feedback to multiple groups can be a significant challenge. This study aimed to investigate the potential of generative artificial intelligence (GenAI) as a tool for providing feedback on students’ research ideas. We employed GenAI tools to provide personalised formative feedback during a small-group activity focused on helping students formulate research plans. The activity was implemented within two university courses designed for working health professionals. Students were grouped into groups of 5–7 students and provided clear instructions for how to prompt the GenAI for feedback on their research ideas. Participants completed an evaluation survey at the end of the activity that assessed frequency of use, perceived value, utility, and overall satisfaction. Half of the participants (n = 64, 85.3

Pharmacy research focuses on increasingly complex health care challenges, requiring various research designs to generate findings that enable us to advance clinical practice. Mixed methods research combines quantitative and qualitative research to provide a deeper and more nuanced understanding of research problems by leveraging the strengths of each method. Researchers may use one method before the other (e.g., explanatory sequential, exploratory sequential) or use both at the same time (e.g., convergent parallel). While some key steps for conducting mixed methods research align with other common methodologies (e.g., clear purpose and rationale, appropriate design and rigor, transparent data collection), mixed methods studies also include a point of interface that demonstrates how the quantitative and qualitative data were utilized to answer the research question. Within pharmacy practice research, mixed methods research may require unique considerations. As examples, pharmacy dispensing systems vary in how specific data fields are collected and reported, and demographic data can be difficult to standardize given different variable definitions and data collection methods. Further, scheduling interviews with patients and providers requires thoughtful scheduling, which may need to be during appointments or working hours. When combining quantitative and qualitative data from various sources, consistent identifiers may also need to be applied to ensure linkage of subjects' data. Given the increasing use of mixed methods in pharmacy practice research, scholars and readers must ensure they understand how and why mixed methods might be utilized. The purpose of this article is to describe mixed methods research, offer special considerations for its use in pharmacy practice research, and detail contemporary challenges and considerations for those interested in conducting or reading mixed methods studies.

Critical theories, such as Critical Race Theory, are a group of theories developed to explicate structural, historical, and social issues that perpetuate inequities and might inform institutional efforts. This study reviewed critical theory use in health professions education with the primary objectives of understanding how and to what extent these theories have been applied. A rapid review was performed in October 2021 with four electronic databases. Scholarship was screened with Covidence based on inclusion (critical theory and health professions education) and exclusion (gray literature, not written in English, not critical theory, not education setting, not peer reviewed) criteria. Data were extracted, charted, and analyzed by three reviewers through Excel, with findings reviewed by the entire research team. A total of 154 pieces of scholarship were included. Most scholarship emerged between 2010 and 2019 (n = 69, 44.8

OBJECTIVE:The purpose of this study was to examine differences in item difficulty, discrimination, and response time for multiple-choice exam items with 3 answer options compared with items with 4 answer options. METHODS:Twenty items were administered on the Biostatistics final exam as a 4-option item from 2017 to 2020 and as a 3-option item from 2021 to 2024. The 3-option items were created by removing the least chosen distractors. Mean item difficulty, discrimination, and response time for each item were collected from aggregate exam performance data provided by Examsoft. Paired t tests were used to compare exam performance metrics between 4-option items and 3-option items. RESULTS:Item difficulty did not differ between 4-option items (89.75 ± 8.45%) and 3-option items (86.45 ± 15.98%, p =.08). However, item discrimination improved from 0.23 ± 0.11 for 4-option items to 0.29 ± 0.09 for 3-option items (p =.03), whereas response time per multiple choice question (MCQ) decreased from 42.50 ± 14.70 s for 4-option items to 38.10 ± 17.78 s for 3-option items (p =.02). CONCLUSION:Three-option MCQs may offer advantages over 4-option MCQs in pharmacy education by reducing time spent per exam item and enhancing item discrimination without impacting item difficulty. Educators and scholars in pharmacy should further explore strategies for optimizing assessment efficiency and effectiveness without comprising psychometric quality.