The Mount Sinai Health System is a hospital network in New York City. It was formed in September 2013 by merging the operations of Continuum Health Partners and the Mount Sinai Medical Center.The Health System is structured around eight hospital campuses, the Icahn School of Medicine at Mount Sinai and Phillips School of Nursing at Mount Sinai Beth Israel. The eight hospitals are: Mount Sinai Beth Israel, Mount Sinai Brooklyn, Mount Sinai Hospital (including Kravis Children's Hospital), Mount Sinai Queens, Mount Sinai Morningside (formerly Mount Sinai St. Luke's), Mount Sinai West (formerly Mount Sinai Roosevelt), New York Eye and Ear Infirmary of Mount Sinai, and Mount Sinai South Nassau.The Health System includes more than 6,600 primary and specialty care physicians and 13 ambulatory surgical centers. It has ambulatory practices throughout the five boroughs of New York City, Westchester County, and Long Island, along with more than 30 affiliated community health centers.In the 2017-2018 fiscal year, the Health System employed more than 39,000 people and the Icahn School of Medicine at Mount Sinai had 33 multidisciplinary research, educational, and clinical institutes. In addition, the Health System reported 3,360 beds among its seven hospitals as well as 136,528 inpatient admissions, 500,901 Emergency Department visits, and more than 14,700 babies delivered.
BACKGROUND:Large language models (LLMs) are increasingly used in health care but remain vulnerable to medical misinformation. We aimed to evaluate how often these models accept or reject fabricated medical content, and how framing that content as a logical fallacy changes results. METHODS:In this cross-sectional benchmarking analysis, we probed 20 LLMs with more than 3·4 million prompts that all contained health misinformation drawn from three sources: public-forum and social-media dialogues, real hospital discharge notes in which we inserted a single false recommendation, and 300 physician-validated simulated vignettes. Logical fallacies-common patterns of flawed reasoning such as appeals to authority, popularity, or emotion-were used to test how rhetorical framing influences model behaviour. Each prompt was posed once in a neutral base form and ten times with a named logical fallacy. For every run we logged susceptibility (model accepts the false claim) and fallacy detection (model flags the rhetoric). FINDINGS:Across all models and corpora, LLMs were susceptible to fabricated data in 50 108 (31·7%) of 158 000 base prompts. Eight of ten fallacy framings significantly reduced or did not change that rate, led by appeal to popularity (susceptibility 11·9%; difference of -19·8 percentage points; p<0·0001); only the slippery-slope prompt (33·9%; difference of 2·2 percentage points; p<0·0001) and the appeal-to-authority prompt (34·6%; difference of 2·9 percentage points; p<0·0001) increased it. Real hospital notes (with fabricated inserted elements) produced the highest susceptibility to the base prompt (46 108 [46·1%] of 100 000), whereas social-media misinformation showed lower base prompt susceptibility (2479 [8·9%] of 28 000). Performance varied by model: GPT models were the least susceptible and most accurate at fallacy detection, whereas others, such as Gemma-3-4B-it, showed 63·6% (5023 of 7900) susceptibility. INTERPRETATION:These results show that LLMs still absorb harmful medical fabrications, especially when phrased in authoritative clinical prose, yet, counter-intuitively, become less vulnerable when the same claims are wrapped in most logical fallacy styles. Therefore, improving safety appears to depend less on model scale and more on fact-grounding and context-aware guardrails. FUNDING:Scientific Computing and Data at Icahn School of Medicine and National Institutes of Health Office of Research Infrastructure.
Accurate staging of unfavorable intermediate- or high-risk prostate cancer (PCa) is essential for treatment decisions. Conventional imaging often fails to detect lymph node, bone, and visceral metastases, and for this purpose 68Ga-prostate-specific membrane antigen (PSMA)-11 PET/CT is clinically used. This prospective, multicenter, International Atomic Energy Agency-supported trial evaluated the accuracy of 68Ga-PSMA-11 PET/CT for initial staging compared with MRI and histopathology and the impact of 68Ga-PSMA-11 PET/CT on determining surgical eligibility. Methods: In a prospective, international study supported by the International Atomic Energy Agency, 775 patients with high-risk or unfavorable intermediate-risk PCa from 12 centers across 11 countries-including low-, middle-, and high-income settings, scheduled for radical prostatectomy based on conventional imaging (including bone scanning and pelvic MRI) underwent 68Ga-PSMA-11 PET/CT before treatment. PET and MRI findings were compared with radical prostatectomy histopathology, and the impact of PET on radical prostatectomy was assessed. Results: 68Ga-PSMA-11 PET/CT detected metastatic disease (M1) in 20.4% of cases, altering management and preventing prostatectomy in 24.0%. The accuracy for seminal vesicle invasion was 90.1% for 68Ga-PSMA-11 PET/CT versus 57.3% for MRI, and for lymph node metastases it was 91.1% for 68Ga-PSMA-11 PET/CT versus 69.7% for MRI. In 13.1% of patients (78/593), there were discordant results between 68Ga-PSMA-11 PET/CT and histopathology. 68Ga-PSMA-11 PET/CT had false-negative lymph node findings in 8.6% of cases, with the most clinically significant being 4.5% of patients incorrectly staged as N0. False-positive lymph node findings at 68Ga-PSMA-11 PET/CT occurred in 4.5% of patients. Conclusion: 68Ga-PSMA-11 PET/CT significantly improves staging accuracy, reducing the indication for prostatectomy and impacting treatment decisions. These findings, from a broad international cohort including low-, middle-, and high-income countries, support the global adoption of 68Ga-PSMA-11 PET/CT into standard staging protocols for high-risk PCa.
Real-world studies based on electronic health records often require manual chart review to derive patients’ clinical phenotypes, a labor-intensive task with limited scalability. Here, we developed and compared computable phenotyping based on rules using the spaCy framework and a Large Language Model (LLM), GPT-4, for sub-phenotyping of patients with Crohn’s disease, considering age at diagnosis and disease behavior. For our rule-based approach, we leveraged the spaCy framework and for the LLM-based approach, we used the GPT-4 model. The underlying data included 49,572 clinical notes and 2204 radiology reports from 584 Crohn’s disease patients. A test set of 280 clinical texts was labeled at sentence-level, in addition to patient-level ground truth data. The algorithms were evaluated based on their recall, precision, specificity values, and F1 scores. Overall, we observe similar or better performance using GPT-4 compared to the rules. On a note-level, the F1 score is at least 0.90 for disease behavior and 0.82 for age at diagnosis, and on patient level at least 0.66 for disease behavior and 0.71 for age at diagnosis. To our knowledge, this is the first study to explore computable phenotyping algorithms based on clinical narrative text for these complex tasks, where prior inter-annotator agreements ranged from 0.54 to 0.98. There is no statistical evidence for a difference to the performance of human experts on this task. Our findings underline the potential of LLMs for computable phenotyping and may support large-scale cohort analyses from electronic health records and streamline chart review processes in the future. Doctors and researchers often need to group patients by specific medical features (called “phenotypes”) to study the disease and improve care. Much of this information is in free-text clinical notes rather than in tabular data. As an example, for patients with Crohn’s disease, the free-text notes can include important details describing the disease course over time, such as bowel narrowings (strictures), abnormal openings (fistulas), problems with the area around the anus, and age at diagnosis. Prior studies show that relying only on structured data, such as codes used to describe particular diseases, often miss these types of details. Reading notes by hand is more accurate but slow and costly. Natural language processing (NLP) is a computational method to automatically read and extract this information from clinical text. We built two NLP approaches and created new sentence-level datasets to test them. Both approaches found complications well, with combining both approaches giving the most balanced results. This method could save time in research and help clinicians flag people who may need extra care. Schmidt et al. develop sentence-level datasets for Crohn’s phenotypes and compare rule-based NLP with GPT-4 to extract disease behavior and age at diagnosis from EHR notes. Both methods achieve high recall on notes; GPT-4 perfectly identifies age at diagnosis and simple ensembles improve precision and enable chart-review prioritization.
BACKGROUND AND OBJECTIVES:General purpose vision-language models (VLMs) demonstrate impressive capabilities, but their opaque training on uncurated internet data poses critical limitations for high-stakes decision making, such as in neurosurgery. We present CNS-Obsidian, a neurosurgical VLM trained on peer-reviewed neurosurgical literature, and demonstrate its clinical utility compared with GPT-4o in a real-world setting. METHODS:We compiled 23 984 articles from Neurosurgery Publications journals, yielding 78 853 figures and captions. Using GPT-4o and Claude Sonnet-3.5, we converted these image-text pairs into 263 064 training samples across 3 formats: instruction fine-tuning, multiple-choice questions, and differential diagnosis. We trained CNS-Obsidian, a fine-tune of the 34-billion parameter Large Language and Visual Assistant-Next model. In a blinded, randomized deployment trial at NYU Langone Health (August 30-November 30, 2024), neurosurgeons were assigned to use either CNS-Obsidian or a Health Insurance Portability and Accountability Act-compliant GPT-4o end point as a diagnostic copilot after patient consultations. Primary outcomes were diagnostic helpfulness and accuracy, assessed through user ratings and presence of the correct diagnosis within the VLM-provided differential, respectively. RESULTS:CNS-Obsidian matched GPT-4o on synthetic questions (76.13% vs 77.54%, P = .235), but only achieved 46.81% accuracy on human-generated questions vs GPT-4o's 65.70% (P < 10-15). In the randomized trial, 70 consultations were evaluated (32 CNS-Obsidian, 38 GPT-4o) from 959 total consults (7.3% utilization). CNS-Obsidian received positive ratings in 40.62% of cases vs 57.89% for GPT-4o (P = .230). Both models included correct diagnosis in approximately 60% of cases (59.38% vs 65.79%, P = .626). CONCLUSION:Domain-specific VLMs trained on curated scientific literature can approach frontier model performance in specialized medical domains despite being orders of magnitude smaller and less expensive to train. This establishes a transparent framework for scientific communities to build specialized artificial intelligence models. However, low clinical utilization suggests chatbot interfaces may not align with specialist workflows, indicating need for alternative artificial intelligence integration strategies.
BACKGROUND:Recombinant factor VIIa has been shown to slow bleeding in patients with intracerebral haemorrhage (ICH), but no haemostatic agent has been shown to improve clinical outcomes. We aimed to evaluate the safety, clinical efficacy, and effect on growth of ICH and intraventricular haemorrhage (IVH) of recombinant factor VIIa in patients with acute spontaneous ICH who were most likely to benefit from treatment with this agent. METHODS:We conducted a multicentre, prospective, double-blind, randomised, placebo-controlled, adaptive, phase 3 trial (FASTEST) at 93 sites across the USA, Japan, Canada, Spain, Germany, and the UK. Adults aged 18-80 years with a spontaneous ICH of 2-60 mL, IVH in less than two-thirds of one lateral ventricle or in less than a third of both lateral ventricles, a Glasgow Coma Scale score of at least 8, no evidence of recent ischaemic stroke or myocardial infarction, no recent use of anticoagulation medication or other structural cause of ICH, and who had been treated with study medication within 2 h of stroke onset or last known well were eligible for inclusion. Patients were randomly assigned (1:1) by a simple randomisation scheme to either 80 μg/kg recombinant factor VIIa (intervention group) or an identical placebo (placebo group), administered intravenously over 2 min. All investigators and participants were masked to allocated group assignment. The primary outcome was functional outcome at 180 days, measured by modified Rankin Scale (mRS; score 0-2, 3, and 4-6) and analysed by intention to treat in all randomly assigned patients. The primary safety outcome was life-threatening thromboembolic events during the first 4 days, assessed in all randomly assigned participants. The secondary aim was change in ICH volume and ICH plus IVH volume between baseline and 24 h of treatment administration. We performed an ordinal logistic regression, adjusted for age, baseline ICH volume, baseline IVH volume, and pre-stroke mRS. Preplanned interim analyses, including adaptive sample size re-estimation and enrichment to a younger subgroup (aged ≤70 years), were also conducted. This trial is registered with ClinicalTrials.gov (NCT03496883) and is closed to new participants. FINDINGS:Between Dec 3, 2021, and Oct 1, 2025, we screened 3288 patients, of whom 626 participants were randomly assigned and included in the intention-to-treat analyses: 298 (48%) in the placebo group and 328 (52%) in the intervention group. 216 (35%) participants were female and 410 (65%) were male, with a mean age of 61 years (SD 12). Mean time from stroke onset to administration of study drug was 100 min (SD 22). The trial met the prespecified stopping criteria for futility at the second interim analysis. There was no differential effect in the primary clinical outcome measure of mRS at 180 days between the intervention group and placebo group (adjusted common odds ratio 1·09 [95% CI 0·79-1·51]; p=0·61). Life-threatening thromboembolic complications within 4 days occurred in 15 (<5%) participants in the intervention group and in four (1%) in the placebo group (relative risk 3·41 [95% CI 1·14-10·15]; p=0·020). Compared with placebo, recombinant factor VIIa was associated with decreased growth of ICH (-3·7 mL [95% CI -5·4 to -1·9]) and of ICH plus IVH growth (-5·2 mL [-7·6 to -2·8]) between baseline and CT scan at 24 h. INTERPRETATION:Recombinant factor VIIa administered within 2 h of ICH onset slowed haematoma growth, but did not improve functional outcomes and showed a small increased risk of life-threatening thromboembolic complications. Further testing of recombinant factor VIIa in patients with the greatest risk of continued bleeding is ongoing. FUNDING:National Institute of Neurological Diseases and Stroke, Japan Agency for Medical Research and Development, and Novo Nordisk.