Background Large language models (LLMs) are increasingly used to generate case-based materials for health professions education (HPE). Previous reviews have examined broad applications in HPE or focused on clinical cases, but none have comprehensively synthesized the generation and evaluation of diverse case-based materials. Objective This study aims to synthesize current evidence on LLM-generated case-based materials in HPE, focusing on generation approaches, evaluation methods, key findings, and reported challenges. Methods This scoping review followed the Joanna Briggs Institute methodology and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) reporting guidelines. Seven databases (PubMed, Embase, Scopus, Web of Science, CINAHL, ERIC, and IEEE Xplore) were searched on May 10, 2026, for studies published from January 1, 2017. Eligible studies were peer-reviewed original research articles, published in English, reporting the use of generative LLMs to create text-based case-based materials for HPE. Data were synthesized using descriptive statistics and narrative synthesis. Results Seventy-two studies were included. Most were published between 2024 and 2026 (67/72, 93%) and focused on medical education (59/72, 82%), with undergraduate students as the most frequently studied population. Case-based multiple-choice questions (MCQs; 28/72, 39%) and clinical cases or vignettes (22/72, 30%) were the most frequently generated materials. Generative Pretrained Transformer-4 (GPT-4) family models were the most commonly used (40/72, 55%), and generation predominantly relied on relatively simple prompting approaches, whereas more advanced strategies, such as few-shot prompting, prompt chaining, or use of external knowledge sources, were rarely adopted. Evaluation primarily relied on expert review, typically involving small panels, while student assessment (25/72, 35%) and psychometric analyses were less common. Overall, LLM-generated materials were frequently judged to be clinically plausible, structurally clear, and reasonably aligned with educational objectives, particularly after refinement. However, consistent limitations included factual inaccuracies, construction flaws, superficial clinical realism, pedagogical misalignment, suboptimal psychometric performance, and bias. No LLM consistently outperformed others, although methodological heterogeneity limits cross-study comparisons. Conclusions This review provides a comprehensive synthesis of the literature on LLM-generated case-based materials in HPE. Unlike previous reviews examining broad educational applications or focusing on clinical cases, it integrates evidence across a broad range of case-based materials while synthesizing generation methods, evaluation approaches, reported findings, and challenges. The evidence suggests that LLMs can support the development of case-based materials but still require expert oversight because of recurring concerns regarding clinical accuracy, educational quality, psychometric performance, clinical realism, and bias. By identifying methodological and evidence gaps, this review provides educators, curriculum developers, and researchers with a structured overview of this rapidly evolving field and may inform the development of more robust generation workflows, more standardized evaluation approaches, and the responsible use of LLM-generated case-based materials in HPE.
BACKGROUND Mobile learning apps are increasingly used in medical education, but cross-national evidence on their longitudinal development and rating-based satisfaction patterns remains limited. OBJECTIVE This study aimed to examine the longitudinal evolution of medical education apps in China and the United States and identify app-related factors associated with positive and negative rating deviations among app store reviewers. METHODS We conducted a longitudinal app store analysis of 467 medical education apps and 232,598 user reviews from major Chinese and US app stores. Joinpoint regression was used to characterize temporal trends in app development. Latent Dirichlet allocation (LDA) topic modeling, with large language model-assisted topic labeling, was used to identify themes in user reviews. Tobit regression, Wald tests, and Kano model classification were used to examine asymmetric associations with positive and negative rating deviations and to classify review topics into functional attributes. RESULTS Among the 232,598 analyzed reviews, 219,105 were rated 4 stars or higher, corresponding to an overall rating-based satisfaction proxy rate of 94.20%. This rating-based satisfaction proxy rate was lower in China than in the United States (128,018/139,416, 91.82% vs 91,087/93,182, 97.75%). Within the 2025-observable analytic cohort, cumulative observable app counts increased significantly in both countries, with average annual percentage changes of 41.69% in China and 41.17% in the United States. The joinpoint analysis identified temporal inflection points within this observable cohort around 2017 and 2021-2022. User concerns also shifted over time. In China, the dominant review theme changed from "Clinical practicality" in 2009-2017 (2735/7385 reviews, 37.03%) to "Practice test function" in 2018-2025 (53,929/132,031 reviews, 40.85%). In the United States, user attention shifted from "Functional availability" in 2009-2017 (20,736/26,118 reviews, 79.39%) to "Cost" in 2018-2025 (27,401/67,064 reviews, 40.86%). Tobit regression and Kano classification showed that factors associated with positive and negative rating deviations differed by country and developmental stage, with essential and charm factors varying across temporal phases. CONCLUSIONS This longitudinal app store analysis revealed distinct but partially converging patterns of observable app development and rating-based satisfaction among app store reviewers in China and the United States. These findings suggest that future medical education apps should prioritize reliable content, technical stability, affordable exam preparation, and personalized learning support. They may also inform the design of future AI-supported medical education platforms while requiring further validation with learning process and outcome data.
BACKGROUND Large language model-based virtual-patient systems score history-taking practice, but whether scores correspond to independently scored examination performance remains untested. Existing evaluations emphasize internal platform metrics or simulated-patient fidelity, leaving criterion-related evidence limited. OBJECTIVE This study tests whether mean preexamination AI-trainer practice scores were associated with performance on 1 local objective structured clinical examination (OSCE) history-taking station, whether the association depended on the complaint-matched training case, and whether a model including the trainer score retained predictive performance in held-out students. METHODS At 1 medical school, second-year students received a required 6-case assignment on the AI-Powered Medical History-Taking Training and Evaluation System (AMTES) and completed an end-of-course OSCE. Each student completed 1 station rated by 1 examiner; interrater and cross-station reliability could not be estimated. The 4 OSCE diagnoses differed from the training diagnoses: 3 shared a presenting-complaint category with an AMTES case, whereas 1 OSCE case had no presenting-complaint match. Platform-to-OSCE record linkage was 474 out of 480 (98.8%); 473 students with analyzable preexamination practice formed the analytic sample. The primary predictor was the mean of normalized scores across practice records time-stamped before each student's examination-day cut-off. Analyses used Spearman correlations, OSCE-case fixed-effects regression, and repeated 5-fold cross-validation. RESULTS OSCE scores (mean 81.5, SD 6.7) were associated with platform practice scores (Spearman ρ=0.388; P<.001). With OSCE-case fixed effects, each 1SD increase in the platform score corresponded to a 2.83-point increase in the OSCE score (95% CI 2.26-3.39). The association was observed in the AMTES-unmatched low back/flank pain stratum (Spearman ρ=0.370; n=108) and with a leave-matched-case-out score (Spearman ρ=0.398 vs 0.370 for the full mean among the same 339 students). Under a single cohort-wide cut-off at the first examination day, the association attenuated but persisted among the same 469 students (Spearman ρ decreased from 0.385 to 0.346; paired difference -0.040, 95% CI -0.065 to -0.018; regression coefficient decreased from 2.83 to 2.68 points per SD). In repeated 5-fold cross-validation (n=471), the platform-only model achieved Spearman ρ=0.374 and R2=0.176. Adding the platform score to the 2 prior physical-examination course scores increased cross-validated R2 from 0.141 to 0.236 (ΔR2=0.096; 95% CI 0.053-0.142). Adding practice volume and case breadth did not improve performance (ΔR2=-0.002). CONCLUSIONS By linking required curricular practice scores to the score on 1 independently scored local OSCE history-taking station, this observational study provides criterion-related evidence for interpreting AMTES scores. The association did not depend on including the complaint-matched training-case score. The scores also added predictive information beyond the 2 prior physical-examination course scores in internal cross-validation. These findings establish neither causal effects of practice nor within-student transfer, and residual confounding by prior ability cannot be excluded. They support further evaluation of AMTES analytics as a candidate low-stakes formative signal, with independent external validation required before operational use.
