Objective: To evaluate Epic IP Insights, an electronic health record integrated generative AI summarizer used across a seven-hospital health system, using a multidimensional assurance framework. Materials and Methods: We developed and validated an agentic hallucination detector that decomposed summaries into atomic content units (ACUs) and verified each against source documentation. We then applied it to remaining summaries to estimate the hallucination rate. We also measured source note utilization, textual and semantic similarity of regenerated summaries, and clinician perceptions through structured evaluations. Results: The EPIC IP Insights generated 706 summaries across 445 encounters and used a mean of 5.8% of available notes. The detector was developed on 30 summaries (2,012 ACUs) and validated on 5 held out summaries (295 ACUs). The detector agreed with physician adjudication on 96.95% ACUs in the validation set. Across 671 remaining summaries containing 40,452 ACUs, the hallucination rate was 11.79% (95% CI, 11.47% to 12.10%). Among 385 regenerated pairs of summaries, 32.7% were textually identical and 37.9% were semantically identical. Clinicians found the tool easy to use (97.8%), while responses were more mixed regarding reliance on the output with little verification (45.9%), expected efficiency gains (45.3%), and frequent use (46.1%). Discussion: Clinicians found the tool easy to use but believed that outputs required verification. Systematic evaluation identified the frequency of hallucinations and limited use of available notes which provide directions for future improvement. Conclusion: Multidimensional assurance frameworks are needed to evaluate the safety, reliability, and consistency of generative AI tools. Keywords generative artificial intelligence; electronic health records; clinical summarization; hallucination detection; AI assurance; large language

Background Chronic Obstructive Pulmonary Disease (COPD) is a leading cause of morbidity and mortality globally. Effective management hinges on early diagnosis, which is often impeded by non-specific symptoms and resource-intensive diagnostic methods. This study assesses the effectiveness of electrocardiograms (ECGs) analysed via deep learning as a tool for early COPD detection. Methods We utilised a Convolutional Neural Network model to analyse ECGs for detecting COPD. The primary outcome was the accuracy of a new clinical COPD diagnosis as determined by ICD codes. Performance was evaluated using Area-Under-the-Curve (AUC) metrics derived by testing against ECGs from a set of holdout patients, ECGs from patients from another hospital, and ECGs of patients with COPD within the UK BioBank (UKBB). Findings We analysed a total of 208,231 ECGs from 18,225 COPD cases, matched to 49,356 controls by age, sex, and race. The model exhibited robust performance across diverse populations with an AUC of 0.80 (0.80-0.80) in internal testing, 0.82 (0.81-0.82) in external validation and 0.75 (0.71-0.78) in the UKBB cohort. Subsequent analyses linked ECG-derived model predictions with spirometry data, and model explainability highlighted P-wave changes as indicative of COPD. Interpretation AI-powered ECG analysis offers a promising path for early COPD detection, potentially facilitating earlier and more effective management. Implementing such tools in clinical settings could significantly enhance COPD screening and diagnostic accuracy, thereby improving patient outcomes and addressing the global health burden of the disease.

Treatment reasoning underpins every therapeutic decision, integrating disease context, comorbidities, medications, contraindications, and evolving biomedical knowledge to select an appropriate therapy. It is inherently iterative: candidates are weighed against many constraints, revised as evidence emerges, and grounded in verifiable sources. Here we introduce ATHENA-R1, an AI agent for treatment reasoning across all FDA approved drugs since 1939, trained by reinforcement learning over a universe of 212 biomedical tools. At each step it identifies missing information, selects and runs relevant tools, and incorporates the evidence. To train it without human-annotated traces, we build a two-level self-learning framework: multi-agent systems construct the tools, tasks, and reasoning trajectories for supervised fine-tuning, then reinforcement learning with scientific feedback rewards reasoning quality (evidence gathering, grounded tool use, logical non-redundancy). Across five benchmarks of 3,168 drug reasoning tasks and 456 patient treatment cases, ATHENA-R1 outperforms language models and tool-use systems, reaching 94.7

BACKGROUND:Chronic Obstructive Pulmonary Disease (COPD) is a leading cause of morbidity and mortality globally. Effective management hinges on early diagnosis, which is often impeded by non-specific symptoms and resource-intensive diagnostic methods. This study assesses the effectiveness of electrocardiograms (ECGs) analysed via deep learning as a tool for early COPD detection. METHODS:We utilised a Convolutional Neural Network model to analyse ECGs for detecting COPD. The primary outcome was the accuracy of a new clinical COPD diagnosis as determined by ICD codes. Performance was evaluated using Area-Under-the-Curve (AUC) metrics derived by testing against ECGs from a set of holdout patients, ECGs from patients from another hospital, and ECGs of patients with COPD within the UK BioBank (UKBB). FINDINGS:We analysed a total of 208,231 ECGs from 18,225 COPD cases, matched to 49,356 controls by age, sex, and race. The model exhibited robust performance across diverse populations with an AUC of 0⋅80 (0⋅80-0⋅80) in internal testing, 0⋅82 (0⋅81-0⋅82) in external validation and 0⋅75 (0⋅71-0⋅78) in the UKBB cohort. Subsequent analyses linked ECG-derived model predictions with spirometry data, and model explainability highlighted P-wave changes as indicative of COPD. INTERPRETATION:AI-powered ECG analysis offers a promising path for early COPD detection, potentially facilitating earlier and more effective management. Implementing such tools in clinical settings could significantly enhance COPD screening and diagnostic accuracy, thereby improving patient outcomes and addressing the global health burden of the disease. FUNDING:This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai and supported by the Clinical and Translational Science Awards (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences; and R01HL167050-02 from the National Heart, Lung, and Blood Institute.

The growing adoption of diagnostic and prognostic algorithms in healthcare has led to concerns about the perpetuation of algorithmic bias against disadvantaged groups of individuals. Deep learning methods to detect and mitigate bias have revolved around modifying models, optimization strategies, and threshold calibration with varying levels of success and tradeoffs. However, there have been limited substantive efforts to address bias at the level of the data used to generate algorithms in healthcare datasets. We create a simple metric (AEquity) that utilizes a learning curve approximation to distinguish and mitigate bias via guided dataset collection or relabeling. We demonstrate this metric in two well-known examples: chest X-rays and healthcare cost utilization, and detect novel biases in the National Health and Nutrition Examination Survey. We demonstrate that utilizing AEquity to guide data-centric collection for each diagnostic finding in the chest radiograph dataset decreased bias by between 29% and 96.5% when measured by differences in area-under-the-curve. When we examined Black patients on Medicaid, at the intersection of race and socioeconomic status, we found that AEquity-based interventions reduced bias across a number of different fairness metrics including overall false negative rate by 33.3% (Bias Reduction Absolute = 1.88 x 10-1; 95% CI (1.4x10-1, 2.5x10-1); Bias Reduction (%) 33.3% (95% CI, 26.6-40.0)), Precision Bias by 7.50x10-2; 95% CI (7.48x10-2, 7.51x10-2); Bias Reduction (%) 94.6% (95% CI, 94.5-94.7%); False Discovery Rate by 94.5% (Absolute Bias Reduction = 3.50x10-2; 95% CI: (3.49x10-2, 3.50x10-2). Similarly, AEquity-guided data collection demonstrates bias reduction of up to 80% on mortality prediction with the National Health and Nutrition Examination Survey (Bias Reduction Absolute = 0.08; 95% CI (0.07, 0.09)). Additionally, we benchmark against balanced empirical risk minimization and calibration and we show that AEquity-guided data collection outperforms both standard approaches. Moreover, we demonstrate that AEquity works on fully connected networks, convolutional neural networks such as ResNet-50, transformer architectures such as on VIT-B-16, an 86 million parameter Vision Transformer, and nonparametric methods such as LightGBM In short, we demonstrate AEquity is a robust tool by applying it to different datasets and algorithms, intersectional analyses and measuring its effectiveness with respect to a range of traditional fairness metrics.

Background: Major Adverse Kidney Events within 30 days (MAKE30) is an important patient-centered outcome for assessing the impact of acute kidney injury (AKI). The existing prediction models for MAKE30 are static and overlook dynamic changes in clinical status. In this study, we introduce ORAKLE, a novel deep-learning model that utilizes evolving time-series data to predict MAKE30, enabling personalized, patient-centered approaches to AKI management and outcome improvement. Methods: We conducted a retrospective study using three publicly available critical care databases: MIMIC-IV, SICdb, and eICU-CRD. Among these, MIMIC-IV was divided into 80% training and 20% internal test sets, whereas SiCdb and eICU-CRD were used as external validation cohorts. Patients with sepsis-3 criteria who developed AKI within 48 hours of intensive care unit admission were identified. Our primary outcome was MAKE30, defined as a composite of death, new dialysis or persistent kidney dysfunction within 30 days of ICU admission. We developed ORAKLE using Dynamic DeepHit framework for time-series survival analysis and its performance against Cox models using AUROC and AUPRC. We further assessed model calibration using Brier score. Results: We analyzed 16,671 patients from MIMIC-IV, 2,665 from SICdb, and 11,447 from eICU-CRD. ORAKLE outperformed the Cox models in predicting MAKE30, achieving AUROCs of 0.84 (95% CI: 0.83-0.86) vs. in MIMIC-IV internal test set 0.80 (95% CI: 0.78-0.82), 0.83 (95% CI: 0.81-0.85) vs. 0.79 (95% CI: 0.77-0.81) in SICdb, and 0.85 (95% CI: 0.84-0.85) vs. 0.81 (95% CI: 0.80-0.82) in eICU-CRD. The AUPRC values for ORAKLE were also significantly better than that of Cox models. The Brier score for ORAKLE was 0.21 across the internal test set, SICdb, and eICU-CRD, suggesting good calibration. Conclusions: ORAKLE is a robust deep-learning model for predicting MAKE30 in critically ill patients with AKI that utilizes evolving time series data. By incorporating dynamically changing time series features, the model captures the evolving nature of kidney injury, treatment effects, and patient trajectories more accurately. This innovation facilitates tailored risk assessments and identifies varying treatment responses, laying the groundwork for more personalized and effective management approaches. ### Competing Interest Statement GNN is a founder of Renalytix, Pensieve, Verici and provides consultancy services to AstraZeneca, Reata, Renalytix, Siemens Healthineer and Variant Bio, serves a scientific advisory board member for Renalytix and Pensieve. He also has equity in Renalytix, Pensieve and Verici. JAK reports receiving consulting fees from Astute Medical/bioMerieux, Astellas, Alexion, Chugai Pharma, Novartis, Mitsubishi Tenabe and GE Healthcare and is a Full-time employee of Spectral Medical. All remaining authors have declared no conflicts of interest. ### Funding Statement This study was supported by NIH grant K08DK131286 (AS). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The study received approval from the Institutional Review Board at the Icahn School of Medicine at Mount Sinai (approval no. 19-00951). I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Publicly available datasets were analyzed in this study. The MIMIC-IV dataset is available at https://physionet.org/content/mimiciv/, the eICU-CRD dataset is available at https://physionet.org/content/eicu-crd/, and the SICdb dataset is available at https://physionet.org/content/sicdb/.

Introduction Strong patient-physician relationships may optimize care in heart failure (HF). Provider consistency is thought to be important for these relationships and potentially allowing for better patient outcomes. As more practices move towards group models, the impact of care provided by a consistent physician on outcomes in HF remains unexplored. Hypothesis We hypothesized that care provided by a consistent physician would be linked to improved outcomes among patients with HF as measured by cardiovascular (CV) hospitalizations and mortality. Methods We retrospectively studied patients with HF treated in an urban health-system from 2015 to 2022. HF was identified upon meeting all 3 of the following criteria: ICD code for HF, prior CV hospitalization, and elevated natriuretic peptide (BNP >100, NTproBNP >400) in the preceding year. We also examined whether patient-physician sex concordance, defined as having consistent care for 75% of visits with a physician of the same sex was associated with outcomes. Varying frequencies were used to define consistency and sex concordance (67%, 75% and 85%) to better assess their relationship with outcomes. We conducted univariate and multivariate Poisson regression analyses on the association of consistent care with CV hospitalizations and mortality. Results A total of 1848 patients met inclusion criteria, with mean age 71.1 ± 14.5, 40% female, and mean follow up time of 620 days. Consistency criteria was met by 1466 (79.3%) patients. Patients without consistency were younger (p<0.001), with more kidney disease (p<0.001), diabetes (p=0.001), and obesity (p=0.002), and more clinic visits on average (9.4 vs 6.8, p<0.001). Physician consistency was associated with significantly fewer CV hospitalizations (Incidence Rate Ratio (IRR) 0.734 [95% CI, 0.688-0.783], p<0.001), after adjusting for age, sex, race, and comorbidities (Fig1). The results were similar with consistency defined as 67% (IRR 0.735 [95% CI, 0.688-0.786], p<0.001), and 85% (IRR 0.733 [95% CI, 0.690-0.778],p<0.001). There was no significant association between consistent care and mortality (p=0.875). Further, we observed that patient-physician sex concordance amongst patients with 75% consistency was also associated with fewer CV hospitalizations (IRR 0.844 [95% CI, 0.797-0.895], p<0.001) but not mortality (HR 1.08; 95%CI 0.889-1.316). Conclusions Among outpatients with HF, receiving care from a consistent physician was associated with fewer CV hospitalizations but not mortality, strengthened by patient-physician sex concordance. Prospective studies are needed to verify these findings and assess how they may play a role in optimizing care delivery in HF.

Major Adverse Kidney Events within 30 days (MAKE30) is an important patient-centered outcome for assessing the impact of acute kidney injury (AKI). Existing prediction models for MAKE30 are static and overlook dynamic changes in clinical status. We introduce ORAKLE, a novel deep-learning model that utilizes evolving time-series data to predict MAKE30, enabling personalized, patient-centered approaches to AKI management and outcome improvement. We conducted a retrospective study using three publicly available critical care databases: MIMIC-IV as the development cohort, and SiCdb and eICU-CRD as external validation cohorts. Patients with sepsis-3 criteria who developed AKI within 48 h of intensive care unit admission were identified. Our primary outcome was MAKE30, defined as a composite of death, new dialysis or persistent kidney dysfunction within 30 days of ICU admission. We developed ORAKLE using Dynamic DeepHit framework for time-series survival analysis and its performance against Cox and XGBoost models. We further assessed model calibration using Brier score. We analyzed 16,671 patients from MIMIC-IV, 2665 from SICdb, and 11,447 from eICU-CRD. ORAKLE outperformed the XGBoost and Cox models in predicting MAKE30, achieving AUROCs of 0.84 (95

This study introduces Glucose Level Understanding and Control Optimized for Safety and Efficacy (GLUCOSE), a distributional offline reinforcement learning algorithm for optimizing insulin dosing after cardiac surgery. Trained on 5228 patients, tested on 920, and externally validated on 649, GLUCOSE achieved a mean estimated reward of 0.0 [–0.07, 0.06] in internal testing and –0.63 [–0.74, –0.52] in external validation, outperforming clinician returns of –1.29 [–1.37, –1.20] and –1.02 [–1.16, –0.89]. In multi-phase human validation, GLUCOSE first showed a significantly lower mean absolute error (MAE) in insulin dosing, with 0.9 units MAE versus clinicians’ 1.97 units (p < 0.001) in internal testing and 1.90 versus 2.24 units (p = 0.003) in external validation. The second and third phases found GLUCOSE’s performance as comparable to or exceeding that of senior clinicians in MAE, safety, effectiveness, and acceptability. These findings suggest GLUCOSE as a robust tool for improving postoperative glucose management.

Rationale:Intravenous fluids are mainstay of management of acute kidney injury (AKI) after sepsis but can cause fluid overload. Recent literature shows that restrictive fluid strategy may be beneficial in some patients with AKI, however, identifying these patients is challenging. Objectives:We aimed to develop and validate a machine learning algorithm to identify patients who would benefit from a restrictive fluid strategy. Methods:We included patients with sepsis who developed AKI within 48 hours of ICU admission and defined restrictive fluid strategy as receiving <500mL fluids within 24 hours after AKI. Our primary outcome was early AKI reversal at 24 hours of AKI onset, and secondary outcomes included sustained AKI reversal and major adverse kidney events by 30 days (MAKE30). We used a causal forest, a machine learning algorithm to estimate individual treatment effects and policy tree algorithm to identify patients who would benefit by restrictive fluid strategy. We developed the algorithm in MIMIC-IV and validated it in SICdb databases. Measurements and Main Results:Among 2,044 patients in the external validation cohort, policy tree recommended restrictive fluids for 66.7%. Among these, patients who received restrictive fluids demonstrated significantly higher rate of early AKI reversal (47.1% vs 31.7%,p=0.004), sustained AKI reversal (28.7% vs 17.5%, p=0.013) and lower rates of MAKE30 (23.0% vs 37.1%, p=0.011). These results were consistent in adjusted analysis. Conclusion:Policy tree based on causal machine learning can identify septic patients with AKI who benefit from a restrictive fluid strategy. This approach needs to be validated in prospective trials.

Importance Increased intracranial pressure (ICP) is associated with adverse neurological outcomes, but needs invasive monitoring. Objective Development and validation of an AI approach for detecting increased ICP (aICP) using only non-invasive extracranial physiological waveform data. Design Retrospective diagnostic study of AI-assisted detection of increased ICP. We developed an AI model using exclusively extracranial waveforms, externally validated it and assessed associations with clinical outcomes. Setting MIMIC-III Waveform Database (2000-2013), a database derived from patients admitted to an ICU in an academic Boston hospital, was used for development of the aICP model, and to report association with neurologic outcomes. Data from Mount Sinai Hospital (2020-2022) in New York City was used for external validation. Participants Patients were included if they were older than 18 years, and were monitored with electrocardiograms, arterial blood pressure, respiratory impedance plethysmography and pulse oximetry. Patients who additionally had intracranial pressure monitoring were used for development (N=157) and external validation (N=56). Patients without intracranial monitors were used for association with outcomes (N=1694). Exposures Extracranial waveforms including electrocardiogram, arterial blood pressure, plethysmography and SpO 2 . Main Outcomes and Measures Intracranial pressure > 15 mmHg. Measures were Area under receiver operating characteristic curves (AUROCs), sensitivity, specificity, and accuracy at threshold of 0.5. We calculated odds ratios and p-values for phenotype association. Results The AUROC was 0.91 (95% CI, 0.90-0.91) on testing and 0.80 (95% CI, 0.80-0.80) on external validation. aICP had accuracy, sensitivity, and specificity of 73.8% (95% CI, 72.0%-75.6%), 99.5% (95% CI 99.3%-99.6%), and 76.9% (95% CI, 74.0-79.8%) on external validation. A ten-percentile increment was associated with stroke (OR=2.12; 95% CI, 1.27-3.13), brain malignancy (OR=1.68; 95% CI, 1.09-2.60), subdural hemorrhage (OR=1.66; 95% CI, 1.07-2.57), intracerebral hemorrhage (OR=1.18; 95% CI, 1.07-1.32), and procedures like percutaneous brain biopsy (OR=1.58; 95% CI, 1.15-2.18) and craniotomy (OR = 1.43; 95% CI, 1.12-1.84; P < 0.05 for all). Conclusions and Relevance aICP provides accurate, non-invasive estimation of increased ICP, and is associated with neurological outcomes and neurosurgical procedures in patients without intracranial monitoring.

Generative Large Language Models (LLMs) hold significant promise in healthcare, demonstrating capabilities such as passing medical licensing exams and providing clinical knowledge. However, their current use as information retrieval tools is limited by challenges like data staleness, resource demands, and occasional generation of incorrect information. This study assessed the potential of LLMs to function as autonomous agents in a simulated tertiary care medical center, using real-world clinical cases across multiple specialties. Both proprietary and open-source LLMs were evaluated, with Retrieval Augmented Generation (RAG) enhancing contextual relevance. Proprietary models, particularly GPT-4, generally outperformed open-source models, showing improved guideline adherence and more accurate responses with RAG. The manual evaluation by expert clinicians was crucial in validating models' outputs, underscoring the importance of human oversight in LLM operation. Further, the study emphasizes Natural Language Programming (NLP) as the appropriate paradigm for modifying model behavior, allowing for precise adjustments through tailored prompts and real-world interactions. This approach highlights the potential of LLMs to significantly enhance and supplement clinical decision-making, while also emphasizing the value of continuous expert involvement and the flexibility of NLP to ensure their reliability and effectiveness in healthcare settings.

Importance. Medical ethics is inherently complex, shaped by a broad spectrum of opinions, experiences, and cultural perspectives. The integration of large language models (LLMs) in healthcare is new and requires an understanding of their consistent adherence to ethical standards. Objective. To compare the agreement rates in answering questions based on ethically ambiguous situations between three frontier LLMs (GPT-4, Gemini-pro-1.5, and Llama-3-70b) and a multi-disciplinary physician group. Methods. In this cross-sectional study, three LLMs generated 1,248 medical ethics questions. These questions were derived based on the principles outlined in the American College of Physicians Ethics Manual. The topics spanned traditional, inclusive, interdisciplinary, and contemporary themes. Each model was then tasked in answering all generated questions. Twelve practicing physicians evaluated and responded to a randomly selected 10% subset of these questions. We compared agreement rates in question answering among the physicians, between the physicians and LLMs, and among LLMs. Results. The models generated a total of 3,744 answers. Despite physicians perceiving the questions' complexity as moderate, with scores between 2 and 3 on a 5-point scale, their agreement rate was only 55.9%. The agreement between physicians and LLMs was also low at 57.9%. In contrast, the agreement rate among LLMs was notably higher at 76.8% (p < 0.001), emphasizing the consistency in LLM responses compared to both physician-physician and physician-LLM agreement. Conclusions. LLMs demonstrate higher agreement rates in ethically complex scenarios compared to physicians, suggesting their potential utility as consultants in ambiguous ethical situations. Future research should explore how LLMs can enhance consistency while adapting to the complexities of real-world ethical dilemmas.

Background: Evidence-based medicine (EBM) is fundamental to modern clinical practice, requiring clinicians to continually update their knowledge and apply the best clinical evidence in patient care. The practice of EBM faces challenges due to rapid advancements in medical research, leading to information overload for clinicians. The integration of artificial intelligence (AI), specifically Generative Large Language Models (LLMs), offers a promising solution towards managing this complexity. Methods: This study involved the curation of real-world clinical cases across various specialties, converting them into .json files for analysis. LLMs, including proprietary models like ChatGPT 3.5 and 4, Gemini Pro, and open-source models like LLaMA v2 and Mixtral-8x7B, were employed. These models were equipped with tools to retrieve information from case files and make clinical decisions similar to how clinicians must operate in the real world. Model performance was evaluated based on correctness of final answer, judicious use of tools, conformity to guidelines, and resistance to hallucinations. Results: GPT-4 was most capable of autonomous operation in a clinical setting, being generally more effective in ordering relevant investigations and conforming to clinical guidelines. Limitations were observed in terms of model ability to handle complex guidelines and diagnostic nuances. Retrieval Augmented Generation made recommendations more tailored to patients and healthcare systems. Conclusions: LLMs can be made to function as autonomous practitioners of evidence-based medicine. Their ability to utilize tooling can be harnessed to interact with the infrastructure of a real-world healthcare system and perform the tasks of patient management in a guideline directed manner. Prompt engineering may help to further enhance this potential and transform healthcare for the clinician and the patient.

Increased intracranial pressure (ICP) >= 15 mmHg is associated with adverse neurological outcomes, but needs invasive intracranial monitoring. Using the publicly available MIMIC-III Waveform Database (2000-2013) from Boston, we developed an artificial intelligence-derived biomarker for elevated ICP (aICP) for adult patients. aICP uses routinely collected extracranial waveform data as input, reducing the need for invasive monitoring. We externally validated aICP with an independent dataset from the Mount Sinai Hospital (2020-2022) in New York City. The AUROC, accuracy, sensitivity, and specificity on the external validation dataset were 0.80 (95% CI, 0.80-0.80), 73.8% (95% CI, 72.0-75.6%), 73.5% (95% CI 72.5-74.5%), and 73.0% (95% CI, 72.0-74.0%), respectively. We also present an exploratory analysis showing aICP predictions are associated with clinical phenotypes. A ten-percentile increment was associated with brain malignancy (OR = 1.68; 95% CI, 1.09-2.60), intracerebral hemorrhage (OR = 1.18; 95% CI, 1.07-1.32), and craniotomy (OR = 1.43; 95% CI, 1.12-1.84; P < 0.05 for all).

Background Artificial intelligence (AI) and large language models (LLMs) can play a critical role in emergency room operations by augmenting decision-making about patient admission. However, there are no studies for LLMs using real-world data and scenarios, in comparison to and being informed by traditional supervised machine learning (ML) models. We evaluated the performance of GPT-4 for predicting patient admissions from emergency department (ED) visits. We compared performance to traditional ML models both naively and when informed by few-shot examples and/or numerical probabilities. Methods We conducted a retrospective study using electronic health records across 7 NYC hospitals. We trained Bio-Clinical-BERT and XGBoost (XGB) models on unstructured and structured data, respectively, and created an ensemble model reflecting ML performance. We then assessed GPT-4 capabilities in many scenarios: through Zero-shot, Few-shot with and without retrieval-augmented generation (RAG), and with and without ML numerical probabilities. Results The Ensemble ML model achieved an area under the receiver operating characteristic curve (AUC) of 0.88, an area under the precision-recall curve (AUPRC) of 0.72 and an accuracy of 82.9%. The naïve GPT-4's performance (0.79 AUC, 0.48 AUPRC, and 77.5% accuracy) showed substantial improvement when given limited, relevant data to learn from (ie, RAG) and underlying ML probabilities (0.87 AUC, 0.71 AUPRC, and 83.1% accuracy). Interestingly, RAG alone boosted performance to near peak levels (0.82 AUC, 0.56 AUPRC, and 81.3% accuracy). Conclusions The naïve LLM had limited performance but showed significant improvement in predicting ED admissions when supplemented with real-world examples to learn from, particularly through RAG, and/or numerical probabilities from traditional ML models. Its peak performance, although slightly lower than the pure ML model, is noteworthy given its potential for providing reasoning behind predictions. Further refinement of LLMs with real-world data is necessary for successful integration as decision-support tools in care settings.

Univariate high-frequency time series are dominant data sources for many medical, economic and environmental applications. In many of these domains, the time series are tied to real-time changes in state. In the intensive care unit, for example, changes and intracranial pressure waveforms can indicate whether a patient is developing decreased blood perfusion to the brain during a stroke, for example. However, most representation learning to resolve states is conducted in an offline, batch-dependent manner. In high frequency time-series, high intra-state and inter-sample variability makes offline, batch-dependent learning a relatively difficult task. Hence, we propose Spatial Resolved Temporal Networks (SpaRTeN), a novel composite deep learning model for online, unsupervised representation learning through a spatially constrained latent space. SpaRTeN maps waveforms to states, and learns time-dependent representations of each state. Our key contribution is that we generate clinically relevant representations of each state for intracranial pressure waveforms.

Univariate high-frequency time series are dominant data sources for many medical, economic and environmental applications. In many of these domains, the time series are tied to real-time changes in state. In the intensive care unit, for example, changes in an electrocardiogram signal can indicate a heart attack, and intracranial pressure waveforms can indicate whether a patient is developing decreased blood perfusion to the brain. However, most representation learning to resolve states is conducted in an offline, batch-dependent manner. In high frequency time-series, high intra-state and inter-sample variability makes offline, batch-dependent learning a relatively difficult task. Hence, we propose Spatial Resolved Temporal Networks (SpaRTeN), a novel composite deep learning model for online, unsupervised representation learning through a spatially constrained latent space. We simultaneously train two distinct blocks: a recurrent neural network ensemble $f_R$ that captures states in high frequency time series, and a spatial block $f_S$ that spatially resolves state changes from the predictions generated by $f_R$. The spatial block $f_S$ identifies the block in $f_R$ that best fits the current state of the time series, and the training procedure for $f_R$ optimizes that block. This procedure corresponds to a minimax framework. When $f_S$ and $f_R$ are deep neural networks, the entire system can be trained via back-propagation. Finally, we demonstrate the application of this framework to online forecasting and interpretable, zero-shot clustering. We compare and demonstrate that SpaRTeN outperforms spectral clustering and a Gaussian mixture model.

Vy, Ha My T.; Coca, Steven G.; Sawant, Ashwin; Sakhuja, Ankit; Gutierrez, Orlando M.; Cooper, Richard; Loos, Ruth J.F.; Horowitz, Carol R.; Do, Ron; Nadkarni, Girish N. Author Information