Preventive care services (PCS) extend life, yet physicians often underprioritize highly effective interventions such as lifestyle modifications (Zhang et al., JAMA Network Open 2020). We evaluated whether large language models (LLMs) replicate and augment physician prioritization of PCS under time constraints. Using Zhang et al.'s validated survey with two patients assessed during long and short visits, we compared seven LLMs with historical physicians. We generated 137 simulated physician personas matching cohort demographics and tested three prompts per model. Primary outcomes were concordance with physician rankings, measured by Spearman correlation, and Consensus-Stratified Agreement (CSA), the proportion of LLM selections rated 4 or higher that matched physician consensus across agreement strata. Secondary outcomes included life-years gained per prioritized choice (LYGPC), consistency, and selectiveness. Augmentation was assessed by having models revise physician rankings under three informative prompts, with delta LYGPC quantifying impact. LLMs closely mirrored physicians (mean Spearman = 0.83, SD = 0.11), with high CSA at extreme agreement ranges (94

Background:The use of large language models (LLMs) is increasing in the medical field; however, LLMs are often subject to "confabulations." Notably, LLMs have vulnerability to adversarial attacks, or fabricated details within prompts, which is concerning given both health misinformation and inadvertent errors in the medical record. This purpose of this study was to determine the effect of adversarial attacks by embedding one fabricated medication into a list of existing medicines. Methods:A total of 250 cases were created, which included 4-6 medications and one fabricated medication (a Pokémon character). Four LLMs (GPT-4o-mini, Gemma-3-27B-IT, Llama-3.3-70B-Instruct, and Qwen3-32B) were tested in triplicate for both dosing information and disease indication with a default prompt, mitigation prompt, and the default prompt with a temperature of 0. If the LLM responded as if the Pokémon were a real medication, it was deemed a confabulation. The primary outcome was the rate of confabulations; exact paired-permutation tests were used to evaluate differences among LLMs and prompting approaches. Results:Confabulation rates for the default and temperature 0 drug dosing prompt ranged from 86-98.8% and 86.9-98.8% across models, respectively, and from 42-95.6% and 41.6-95.5% for the indication prompt. Incorporating the mitigation prompt substantially reduced confabulation rates to 8.3-76.3% (dosing) and 1.7-28.3% (indication). The best-performing model, Llama-3.3-70B-Instruct, demonstrated confabulation rates spanning 1.7-91.9% (p<0.001). Conclusions:LLMs are susceptible to adversarial attacks, especially with medications. Further model improvement is imperative before LLMs are considered safe and reliable for routine use in the medical field.

This comparative effectiveness research assesses the performance of newer open-source large language models (LLMs) with that of closed-source proprietary large LLMs.

Neutrophils have vital proinflammatory protective functions, but gene expression changes in neutrophils found in inflamed tissues suggest additional proresolving effects. We identified a neutrophil subset with a distinct phenotype and function that emerges in mouse lungs during resolution of injury. These resolution-phase neutrophils increased expression of Siglec-F (sialic acid–binding Ig-like lectin F), Alox15 (12/15-lipoxygenase), and Csf1 ( colony-stimulating factor 1 ). Siglec-F + neutrophils promoted macrophage differentiation and produced specialized proresolving mediators that accelerated injury resolution. Neutrophil depletion hindered lung epithelial catabatic responses, whereas adoptive transfer of Siglec-F + neutrophils accelerated restitution of lung epithelial cells. Transforming growth factor–β (TGF-β) and granulocyte-macrophage colony-stimulating factor (GM-CSF), acting via activator protein-1 (AP-1)/Jun, promoted expression of Siglec-F in mouse neutrophils and ALOX15 in mouse and human neutrophils. In patients with respiratory failure, ALOX15 + neutrophils were present in the bronchoalveolar lavage samples, and their frequency correlated with improved oxygenation. Thus, Siglec-F + ALOX15 + proresolving neutrophils contribute to tissue injury responses.

Background Clinical problem-solving requires processing of semantic medical knowledge, such as illness scripts, and numerical medical knowledge of diagnostic tests for evidence-based decision-making. As large language models (LLMs) show promising results in many aspects of language-based clinical practice, their ability to generate nonlanguage evidence-based answers to clinical questions is inherently limited by tokenization. Objective This study aimed to evaluate LLMs’ performance on two question types: numeric (correlating findings) and semantic (differentiating entities), while examining differences within and between LLMs in medical aspects and comparing their performance to humans. Methods To generate straightforward multichoice questions and answers (Q and As) based on evidence-based medicine (EBM), we used a comprehensive medical knowledge graph (containing data from more than 50,000 peer-reviewed studies) and created the EBM questions and answers (EBMQAs). EBMQA comprises 105,222 Q and As, categorized by medical topics (eg, medical disciplines) and nonmedical topics (eg, question length), and classified into numerical or semantic types. We benchmarked a dataset of 24,000 Q and As on two state-of-the-art LLMs, GPT-4 (OpenAI) and Claude 3 Opus (Anthropic). We evaluated the LLM’s accuracy on semantic and numerical question types and according to sublabeled topics. In addition, we examined the question-answering rate of LLMs by enabling them to choose to abstain from responding to questions. For validation, we compared the results for 100 unrelated numerical EBMQA questions between six human medical experts and the two language models. Results In an analysis of 24,542 Q and As, Claude 3 and GPT-4 performed better on semantic Q and As (68.7%, n=1593 and 68.4%, n=1709), respectively. Then on numerical Q and As (61.3%, n=8583 and 56.7%, n=12,038), respectively, with Claude 3 outperforming GPT-4 in numeric accuracy ( P <.001). A median accuracy gap of 7% (IQR 5%‐10%) was observed between the best and worst sublabels per topic, with different LLMs excelling in different sublabels. Focusing on Medical Discipline sublabels, Claude 3 performed well in neoplastic disorders but struggled with genitourinary disorders (69%, n=676 vs 58%, n=464; P <.0001), while GPT-4 excelled in cardiovascular disorders but struggled with neoplastic disorders (60%, n=1076 vs 53%, n=704; P =.0002). Furthermore, humans (82.3%, n=82.3) surpassed both Claude 3 (64.3%, n=64.3; P <.001) and GPT-4 (55.8%, n=55.8; P <.001) in the validation test. Spearman correlation between question-answering and accuracy rate in both Claude 3 and GPT-4 was insignificant (ρ=0.12, P =.69; ρ=0.43, P =.13). Conclusions Both LLMs excelled more in semantic than numerical Q and As, with Claude 3 surpassing GPT-4 in numerical Q and As. However, both LLMs showed inter- and intramodel gaps in different medical aspects and remained inferior to humans. In addition, their ability to respond or abstain from answering a question does not reliably predict how accurately they perform when they do attempt to answer questions. Thus, their medical advice should be addressed carefully.

Differential diagnosis is an iterative process that integrates patient information with broader medical knowledge. Clinical case series such as the NEJM Clinicopathologic Conferences (CPCs), published continuously since 1923, feature expert physicians who demonstrate diagnostic reasoning to peers, and have been used for decades to evaluate AI. However, prior AI evaluations have largely focused on final diagnostic accuracy rather than nuanced clinical reasoning. Here, we introduce Dr. CaBot, an agentic AI system that emulates an expert diagnostician by generating written and narrated slide-based presentations from an initial case description alone. CaBot recently generated the first AI diagnosis published in the 100+ year history of the NEJM CPCs. In blinded evaluations, physicians misclassified the source of the differential (CaBot vs. physician-written) in 46/62 (74

The rapid emergence of artificial intelligence (AI), including generative large language models, offers transformative opportunities in medical education. This proliferation has generated numerous speculative discussions about AI's promise but has been limited in delivering a comprehensive analysis to distinguish evidence-based utility from hype while identifying context-specific limitations.In this first part of a 2-part innovation report, commissioned by the Josiah Macy Jr. Foundation to inform the discussions at a conference on AI in medical education, the authors synthesize the landscape of AI in medical education, underscoring both its potential advantages and inherent challenges. To map the AI landscape, they reviewed 455 articles that targeted 5 medical education domains: (1) admissions, (2) classroom-based learning and teaching, (3) workplace-based learning and teaching, (4) assessment, feedback, and certification, and (5) program evaluation and research.In admissions, AI-driven strategies facilitated holistic applicant reviews through predictive modeling, natural language processing, and large language model-based chatbots. Preclinical learning benefited from AI-powered virtual patients and curriculum design tools that managed expanding medical knowledge and supported robust student practice. Within clinical learning, AI aided diagnostic and interpretive processes, prompting medical education curricula to demand relevant AI competency and literacy frameworks. A few studies reported that assessment and feedback processes became more efficient through automated grading and advanced analytics, which reduced faculty workload and offered timely, targeted feedback. Program evaluation and research gained additional insights using AI on careers, diversity, and performance metrics of faculty and learners, improving resource allocations and guiding evidence-based approaches.Despite these possibilities, bias in AI algorithms, concerns about transparency, inadequate ethical guidelines, and risks of over-reliance highlighted the need for cautious, informed AI implementation. By mapping AI tasks to medical education applications, the authors provide a framework for understanding and leveraging AI's potential while addressing technical, ethical, and human-factor complexities in this evolving field.

Diagnostic errors pose a significant public health challenge, affecting nearly 800,000 Americans annually, with even higher rates globally. In the ICU, these errors are particularly prevalent, leading to substantial morbidity and mortality. The clinical reasoning process aims to reduce diagnostic uncertainty and establish a plausible differential diagnosis but is often hindered by cognitive load, patient complexity, and clinician burnout. These factors contribute to cognitive biases that compromise diagnostic accuracy. Emerging technologies like large language models (LLMs) offer potential solutions to enhance clinical reasoning and improve diagnostic precision. In this perspective article, we explore the roles of LLMs, such as GPT-4, in addressing diagnostic challenges in critical care settings through a case study of a critically ill patient managed with LLM assistance.

A seminal paper published by Ledley and Lusted in 1959 introduced complex clinical diagnostic reasoning cases as the gold standard for the evaluation of expert medical computing systems, a standard that has held ever since. Here, we report the results of a physician evaluation of a large language model (LLM) on challenging clinical cases against a baseline of hundreds of physicians. We conduct five experiments to measure clinical reasoning across differential diagnosis generation, display of diagnostic reasoning, triage differential diagnosis, probabilistic reasoning, and management reasoning, all adjudicated by physician experts with validated psychometrics. We then report a real-world study comparing human expert and AI second opinions in randomly-selected patients in the emergency room of a major tertiary academic medical center in Boston, MA. We compared LLMs and board-certified physicians at three predefined diagnostic touchpoints: triage in the emergency room, initial evaluation by a physician, and admission to the hospital or intensive care unit. In all experiments–both vignettes and emergency room second opinions–the LLM displayed superhuman diagnostic and reasoning abilities, as well as continued improvement from prior generations of AI clinical decision support. Our study suggests that LLMs have achieved superhuman performance on general medical diagnostic and management reasoning, fulfilling the vision put forth by Ledley and Lusted, and motivating the urgent need for prospective trials.

This cross-sectional study assesses the ability of a large language model to process medical data and display clinical reasoning compared with the ability of attending physicians and residents.

Clinical problem-solving requires processing of semantic medical knowledge such as illness scripts and numerical medical knowledge of diagnostic tests for evidence-based decision-making. As large language models (LLMs) show promising results in many aspects of language-based clinical practice, their ability to generate non-language evidence-based answers to clinical questions is inherently limited by tokenization. Therefore, we evaluated LLMs' performance on two question types: numeric (correlating findings) and semantic (differentiating entities) while examining differences within and between LLMs in medical aspects and comparing their performance to humans. To generate straightforward multi-choice questions and answers (QAs) based on evidence-based medicine (EBM), we used a comprehensive medical knowledge graph (encompassed data from more than 50,00 peer-reviewed articles) and created the "EBMQA". EBMQA contains 105,000 QAs labeled with medical and non-medical topics and classified into numerical or semantic questions. We benchmarked this dataset using more than 24,500 QAs on two state-of-the-art LLMs: Chat-GPT4 and Claude3-Opus. We evaluated the LLMs accuracy on semantic and numerical question types and according to sub-labeled topics. For validation, six medical experts were tested on 100 numerical EBMQA questions. We found that both LLMs excelled more in semantic than numerical QAs, with Claude3 surpassing GPT4 in numerical QAs. However, both LLMs showed inter and intra gaps in different medical aspects and remained inferior to humans. Thus, their medical advice should be addressed carefully.

Vitamin D possesses immunomodulatory functions and vitamin D deficiency has been associated with the rise in chronic inflammatory diseases, including asthma (Litonjua and Weiss, 2007). Vitamin D supplementation studies do not provide insight into the molecular genetic mechanisms of vitamin D-mediated immunoregulation. Here, we provide evidence for vitamin D regulation of two human chromosomal loci, Chr17q12-21.1 and Chr17q21.2, reliably associated with autoimmune and chronic inflammatory diseases. We demonstrate increased vitamin D receptor ( Vdr ) expression in mouse lung CD4+ Th2 cells, differential expression of Chr17q12-21.1 and Chr17q21.2 genes in Th2 cells based on vitamin D status and identify the IL-2/Stat5 pathway as a target of vitamin D signaling. Vitamin D deficiency caused severe lung inflammation after allergen challenge in mice that was prevented by long-term prenatal vitamin D supplementation. Mechanistically, vitamin D induced the expression of the Ikzf3 -encoded protein Aiolos to suppress IL-2 signaling and ameliorate cytokine production in Th2 cells. These translational findings demonstrate mechanisms for the immune protective effect of vitamin D in allergic lung inflammation with a strong molecular genetic link to the regulation of both Chr17q12-21.1 and Chr17q21.2 genes and suggest further functional studies and interventional strategies for long-term prevention of asthma and other autoimmune disorders.

BACKGROUND:Large language models (LLMs) such as GPT-4 hold great promise as transformative tools in health care, ranging from automating administrative tasks to augmenting clinical decision making. However, these models also pose a danger of perpetuating biases and delivering incorrect medical diagnoses, which can have a direct, harmful impact on medical care. We aimed to assess whether GPT-4 encodes racial and gender biases that impact its use in health care. METHODS:Using the Azure OpenAI application interface, this model evaluation study tested whether GPT-4 encodes racial and gender biases and examined the impact of such biases on four potential applications of LLMs in the clinical domain-namely, medical education, diagnostic reasoning, clinical plan generation, and subjective patient assessment. We conducted experiments with prompts designed to resemble typical use of GPT-4 within clinical and medical education applications. We used clinical vignettes from NEJM Healer and from published research on implicit bias in health care. GPT-4 estimates of the demographic distribution of medical conditions were compared with true US prevalence estimates. Differential diagnosis and treatment planning were evaluated across demographic groups using standard statistical tests for significance between groups. FINDINGS:We found that GPT-4 did not appropriately model the demographic diversity of medical conditions, consistently producing clinical vignettes that stereotype demographic presentations. The differential diagnoses created by GPT-4 for standardised clinical vignettes were more likely to include diagnoses that stereotype certain races, ethnicities, and genders. Assessment and plans created by the model showed significant association between demographic attributes and recommendations for more expensive procedures as well as differences in patient perception. INTERPRETATION:Our findings highlight the urgent need for comprehensive and transparent bias assessments of LLM tools such as GPT-4 for intended use cases before they are integrated into clinical care. We discuss the potential sources of these biases and potential mitigation strategies before clinical implementation. FUNDING:Priscilla Chan and Mark Zuckerberg.

Learning Objectives: 1) Understand real world presentation of Giant Cell Arteritis (GCA).2) Identify types of bias leading to a delayed GCA diagnosis.Case Report: A 72-year-old male presented with multiple complaints, including scrotal pain, transient vision loss, fatigue, weight loss and new headaches.ED work-up showed leukocytosis and aortic dilation on imaging.Clinicians recommended further work-up for possible infection but did not raise concern for GCA.The patient declined admission but re-presented 2 days later endorsing similar symptoms.Ultrasound showed epididymitis for which he received Bactrim.He presented to clinic several days later with "confusion" which was attributed to Bactrim.The following week, he represented with scrotal pain and confusion.Ultrasound showed improving epididymitis, but his symptoms were attributed to epididymitis, and he was prescribed cefpodoxime.Three days later, he re-presented with worsening headaches, temporal swelling, and elevated CRP, prompting admission.He underwent temporal artery biopsy which confirmed GCA.Discussion: GCA is the most common systemic vasculitis and should be considered for patients over 50 who have new headaches, visual disturbances, and vascular abnormalities.Patients may also present with confusion.Prompt identification and treatment with corticosteroids is imperative as untreated GCA can progress to blindness.Despite his age and multiple complaints consistent with GCA, this diagnosis was not initially considered.His presentation was attributed to epididymitis, which interestingly has been associated with GCA.This patient is emblematic of broader issues in prompt GCA diagnosis, which takes a mean of 9 weeks from symptom onset.Heuristics are a helpful tool for physicians to make diagnoses.However, using mental shortcuts may have played a role in this delay, as GCA is more common in females.Anchoring bias may also have contributed.This patient endorsed persistent scrotal pain and was diagnosed with epididymitis, which may have clouded his additional complaints.