Sepsis-related myocardial injury (SRMI) is associated with high mortality, but the transportability of prediction models across heterogeneous cohorts remains uncertain. We developed and externally validated a machine learning model for in-hospital mortality prediction in SRMI and evaluated its discrimination, calibration, interpretability, and distributional shift across cohorts. Using MIMIC-IV for development and MIMIC-III and a local cohort for external validation (n = 1,226, 962, and 280, respectively), we compared eight classification algorithms. A two-stage feature-selection strategy, recursive feature accumulation and sequential elimination with incremental feature evaluation, ranked 31 candidate variables to derive a parsimonious predictor set. The selected model was evaluated for discrimination, precision-recall performance, calibration, clinical utility, and risk stratification. Logistic recalibration, population stability index (PSI) analysis, and SHapley Additive exPlanations (SHAP) were used to assess probability alignment, cross-cohort distributional shift, and model interpretability. Feature selection reduced 31 candidate variables to 8 predictors with little loss of cross-validated performance (AUC, 0.819 with 31 variables versus 0.810 with 8 variables). The selected random forest model achieved an AUC of 0.819 (bootstrap 95% CI, 0.787-0.848) and a PR-AUC of 0.702 (0.657-0.751) in MIMIC-III, and an AUC of 0.758 (0.693-0.816) and a PR-AUC of 0.525 (0.435-0.630) in the local cohort. Recalibration reduced the Brier score from 0.162 to 0.147 in MIMIC-III and from 0.233 to 0.196 in the local cohort. PSI indicated minimal distributional shift between MIMIC-IV and MIMIC-III but substantial shift in the local cohort, particularly for SOFA score, respiratory rate, and prothrombin time. SHAP identified prothrombin time, invasive ventilation, lactate, temperature, and urine output as major contributors to model predictions. In two external cohorts, an 8-variable random forest model showed moderate discrimination for mortality prediction in SRMI. The evaluation of external performance, calibration, distributional shift, and SHAP-based explanations provides an assessment of model transportability. Prospective validation is needed before clinical deployment.
Systematic reviews of randomized controlled trials (RCTs) support evidence-based clinical advancement. Clinical trial reporting guidelines, including CONSORT, establish common data elements that support systematic synthesis of published trial outcomes. However, the rapid rise of digital health technologies presents new challenges for reporting guidelines, and substantial real-world reporting heterogeneity has been observed. Therefore, machine-assisted evaluation of reporting adherence offers a pathway for more agile information management to accommodate the diverse and disruptive nature of digital health technology-enabled RCTs. Here, we propose an accelerated retrieval-to-abstraction literature synthesis framework to conduct article screening, text and table data preprocessing, and full-text review using an unmodified pretrained large language model. We apply this methodology in two digital health technology-enabled RCT case studies evaluating adherence to technology and digital health equity guideline reporting items. Large language model performance was on par with inter-annotator agreement F1 scores (≥0.9) in both screening and information-extraction tasks. These findings support the scalability and rigor of accelerated evidence synthesis and demonstrate the importance of common reporting elements across digital health technology-enabled RCTs.
Large language models (LLMs) such as GPT-4 and DeepSeek have shown potential in medical education and assessment. In China, the Chinese National Medical Licensing Examination (CNMLE) has become a critical benchmark for evaluating LLMs’ professional knowledge and reasoning ability in non-English contexts. This scoping review aims to synthesize research evaluating LLM performance on the CNMLE and related Chinese medical examinations, identifying performance trends, methodological gaps, and future directions. A scoping review was conducted in accordance with PRISMA-ScR guidelines. PubMed and Web of Science were searched on June 12, 2025, using terms related to LLMs and Chinese medical examinations. Studies were included if they evaluated any LLM on national-level Chinese medical exams and reported performance metrics. Two reviewers screened and extracted data. Study quality was assessed using a 12-item checklist covering dataset characteristics, LLM setup, and evaluation methods. Fisher’s exact test was used to assess differences in the passing rates of different LLMs. 14 studies were included, covering 51 evaluation records across 8 types of Chinese medical examinations, including CNMLE and specialty exams such as critical care, radiation oncology, and ultrasound medicine. Exam years ranged from 2017 to 2024, with a shift toward using recent exams. A total of 9 LLMs were evaluated, including GPT-3.5, GPT-4, GPT-4o, ERNIE, DeepSeek-R1, Qwen-72B, Baichuan2-7B, Baichuan2-13B and DISC-MedLLM. GPT-4o and DeepSeek-R1 achieved the highest scores of 552 (92.00%) and 523 (87.20%) in the 2024 CNMLE, respectively, both surpassing the passing threshold. In an exploratory comparison, GPT-4 achieved a higher pass rate than GPT-3.5 with a statistically significant difference, although inconsistencies in datasets, model transparency, and evaluation criteria persist across the included studies. LLMs show improving performance on Chinese medical exams, especially newer models. Nevertheless, future research should prioritize standardized benchmarks, multimodal capabilities, and transparent evaluation to ensure meaningful clinical relevance and educational value.
Electronic Health Records (EHRs) are pivotal for improving healthcare data quality and service delivery. Despite the introduction of the Impilo EHR system at St Anne’s Mission Hospital in Brunapeg, utilisation remains low. This study aimed to explore healthcare workers’ (HCWs) experiences and identify the key barriers and facilitators to Impilo’s utilisation. A descriptive qualitative study was conducted at St Anne’s Mission Hospital. In-depth interviews were conducted with 20 HCWs authorised to use Impilo, selected via purposive sampling until thematic saturation was achieved. Thematic analysis, following the Braun and Clarke approach, was used to analyse the data. Trustworthiness was ensured through multi-coder analysis, the maintenance of an audit trail, and thick descriptive reporting. HCWs reported positive experiences related to ease of data access, improved communication, and better inventory management. However, significant barriers undermine utilisation. The analysis revealed three key themes: (1) Benefits of Impilo, (2) Multifaceted Challenges, and (3) Suggested Improvements. Challenges included pervasive technical and infrastructural issues (e.g., frequent network downtime, lack of offline functionality), human resource constraints leading to dual documentation and increased workload, and subsequent compromised data quality due to inconsistent entry. The underutilisation of the Impilo system is not due to a lack of perceived value but is primarily driven by remediable barriers related to infrastructure, training, and workflow design issues. For successful implementation, interventions must be multifaceted, addressing technical reliability (e.g., offline functionality), human capacity (continuous training and support), and organisational processes (dedicated data clerks and inclusive system redesign). This study provides a framework for improving EHR adoption in similar low-resource rural settings.
The 12-lead ECG is central to the evaluation of patients with suspected coronary artery disease (CAD) but has limited diagnostic accuracy for identifying significant obstructive disease prior to elective coronary angiography. We previously developed an artificial intelligence–enhanced ECG (AI-ECG) model for detecting acute coronary occlusion. Whether this model generalizes to identifying angiographically significant CAD in symptomatic patients undergoing elective evaluation remains unknown. In this prospective cohort study, symptomatic outpatients undergoing elective coronary angiography at a tertiary center received standard 12-lead ECGs before the procedure. ECGs were analyzed offline using a previously validated AI-ECG model originally developed for acute coronary occlusion. Patients were categorized as low or intermediate-to-high risk using predefined thresholds. The primary outcome was angiographically significant CAD (≥70% stenosis in a major epicardial vessel or ≥50% stenosis of the left main coronary artery). Among 363 patients (age 58.9 ± 11 years; 55.1% male), 36.1% had angiographically significant CAD. The AI-ECG classified 56% as low risk and 44% as intermediate-to-high risk. After adjustment for conventional clinical characteristics, intermediate-to-high risk remained independently associated with significant CAD (OR 3.12, 95% CI 1.88–5.20; p < 0.001). Model discrimination was good (AUROC 0.79, 95% CI 0.74–0.84), with 60% precision at 80% recall. In summary, the ECG-SMART AI score demonstrated good diagnostic performance for identifying angiographically significant CAD in symptomatic patients undergoing elective coronary angiography. These findings support the generalizability of a previously developed AI-ECG model beyond acute coronary occlusion and suggest that AI-enhanced ECG analysis may complement existing clinical assessment by providing additional noninvasive information to refine pretest probability during elective evaluation.
