
Abstract Objectives To develop and externally validate Prognostic Estimator of RiSk for prolonged ICU STay (PERSIST), a model that continuously updates the probability of an intensive care unit (ICU) stay longer than 7 days during the first week of ICU admission. Materials and Methods We developed PERSIST, an XGBoost model, on 49 255 adult ICU stays from the Mount Sinai Health System (MSHS; 2021-2025) and externally validated it in 72 156 Medical Information Mart for Intensive Care–IV (MIMIC-IV) stays (2008-2019). Sixty routinely available electronic health record (EHR) features were aggregated hourly, with predictions every 4 hours from preceding data, benchmarked against L2-regularized logistic regression. We assessed discrimination, calibration, net benefit, SHapley Additive exPlanations attributions, and subgroup performance. Results Prolonged stay occurred in 16.1% of MSHS and 11.1% of MIMIC-IV admissions. PERSIST achieved macroaveraged areas under the receiver operating characteristic curve of 0.80 (95% confidence interval [CI], 0.79-0.81) and 0.78 (0.77-0.78), respectively, compared with 0.75 (0.73-0.76) and 0.74 (0.73-0.74) for logistic regression. Corresponding areas under the precision-recall curve were 0.73 and 0.68. Brier scores were 0.14 and 0.13. Across modeled thresholds, PERSIST provided greater net benefit than treat-all and treat-none strategies. Discrimination was similar across reported age, sex, and racial groups. Discussion Prognostic Estimator of RiSk for prolonged ICU STay transported across 2 health systems differing in era, case mix, and outcome prevalence, and outperformed logistic regression under identical conditions. Net benefit was greatest during the early ICU days, when few patients have declared a prolonged trajectory and a risk estimate can still inform planning. Conclusion Continuously updated, EHR-based prediction of prolonged ICU stay is feasible and generalizable. Prospective evaluation should determine whether it improves ICU care coordination and capacity planning.
Abstract Objectives Personalized or N-of-1 cross-over controlled trials evaluate the effectiveness of an intervention within a single patient. Though data from personalized N-of-1 trials are used to inform individual treatment decisions, personalized trial data can also be aggregated to provide insights about larger populations and improve quality of population treatment decisions. Our goal is to develop a publicly available phenotyping Wizard to conduct analyses using pooled data from multiple series of personalized N-of-1 trials. Materials and methods The current Wizard integrates data from 2 series of personalized, N-of-1 trials of yoga vs massage for improving chronic lower back pain (N = 57) and active vs sham light therapy for improving fatigue (N = 60). This online Wizard aggregates text-message ecological momentary assessment data, Fitbit wearable activity tracker data, and validated survey instruments. Results The Wizard allows for the public and research audiences to access de-identified data from 117 personalized N-of-1 trials with up to 14 weeks of data for each trial. Users of the Wizard can view raw data, create plots of study measures over time, and use previously established analytic methods for N-of-1 trials to examine pooled and sub-group treatment effects. Discussion and Conclusion Data from a personalized N-of-1 trial is seldom analyzed after it has been used to inform an individual patient’s treatment. This Wizard provides an interactive method for visualizing and analyzing aggregated standardized data taken from personalized trials. This platform encourages the use of personalized trial data to explore group phenotypes for treatment response.
Abstract Objectives The Code Map Services/Common Data Model Harmonization Phase III (CDMH-III) project aims to create services allowing researchers to streamline and establish standardized model-to-model/data standards mappings and transformation rules leveraging common data models (CDMs) and common data elements (CDEs) to address the challenge of repeated mapping efforts. Materials and Methods The project team developed CDM transformation services, improved mappings, and semiautomated transformation rule creation. Activities included registration of CDMs and CDEs, development of application programming interfaces, implementation of extract-transform-load workflows, validation of mappings through subject matter expert review, and evaluation using Food and Drug Administration clinical use case involving real-world data. Transformation pathways were established among Health Level Seven Fast Healthcare Interoperability Resources US Core, and Clinical Data Interchange Standards Consortium study data tabulation model. Results Data mappings and transformation rules have been created and stored in the Cancer Data Standards Registry and Repository, a highly curated metadata repository with open accessibility for all users. A workflow architecture was developed to illustrate all the steps in a data acquisition, transformation, and submission process. Discussion The CDMH-III project improves interoperability across clinical data models by enabling automated, standardized data transformations through Code Map Services, allowing researchers to access and analyze data from diverse sources. This supports higher quality, more transparent, and reproducible research while facilitating the use and regulatory submission of real-world data across health care and clinical research systems. Conclusion Common Data Model Harmonization Phase III advances interoperability by providing standardized, semantically driven mapping services that enable automated, reliable data transformation across CDMs and health-care standards.
Abstract Objectives To determine whether statewide health information exchange (HIE) data can support practice-level prediction of short-term psychiatric emergency presentation, and to compare a locally trained model with general-purpose clinical risk scores. Materials and Methods Retrospective cohort of patients at 1 Montana psychiatry practice with records in the Big Sky Care Connect statewide HIE. Structured HIE data were linked to zip code-level Social Vulnerability Index and Area Deprivation Index measures. Each observation was a patient snapshot. Logistic regression, random forest, and LightGBM models were tuned with 5 × 5 nested cross-validation and evaluated on a temporally held-out test set. The primary outcome was psychiatric or substance-related emergency department (ED) presentation within 30 days. Discrimination was compared with a parsimonious clinical baseline and the LACE and Elixhauser indices. Results The test set contained 16 469 snapshots, of which 343 (2.1%) preceded a qualifying ED presentation (102 visits among 68 patients). A random forest model achieved area under the receiver-operator characteristic curve 0.85 (95% CI, 0.75-0.91). At a threshold targeting one-third recall, precision was 0.29 (0.12-0.44) and number needed to evaluate 3.4 (2.3-8.1). Discrimination exceeded the clinical baseline (Δ area under the curve 0.10), LACE (0.19), and Elixhauser (0.27); all P values less than .001. Discussion Structured HIE data, without clinical notes, supported a practice-specific model that outperformed general-purpose scores such as LACE and Elixhauser. Wide CIs reflect a small event count and constrain point-estimate claims. Conclusion health information exchange infrastructure may allow participating practices to build locally calibrated psychiatric risk models. Prospective study of clinical utility is needed.
Abstract Objective Extract infant breast milk feeding status from electronic health record clinical notes to support observational drug-safety studies. Materials and Methods Infant notes (19 489; 228 infants) were weakly labeled by large language model (LLM) and validated on 1045 manually reviewed notes. Using these labels, we built a rule-based medspaCy classifier from a curated feeding term set and compared it with ClinPhen and fine-tuned Bidirectional Encoder Representations from Transformers (BERT) document classifiers. Maternal postpartum notes (23 mothers; 616 partitions) were manually labeled to compare BERT with prompt-based language models. Results LLM labels achieved 99.62% accuracy (macro F1 99.32%). medspaCy achieved 94.36% accuracy and 90.40% balanced accuracy (macro F1 0.90) and was applied to 52 278 mother–infant dyads. On maternal notes, the prompt-based model improved Positive-class F1 vs BERT (0.96 vs 0.82). Discussion LLMs were most effective for scalable labeling; deterministic rules supported transparent large-scale inference. Conclusion Weak supervision plus lightweight rules can convert sparse breastfeeding documentation into structured variables at-scale.