In the sprawling city of Rio de Janeiro, accessing health care can be challenging, even when services are available. A new initiative aims to change that by using AI to proactively connect people to public services. In this News and Perspectives article, JMIR Correspondent Luke Taylor reports on the Connect Rio program, the second story in his two-part series on how AI is helping make health care more proactive in Brazil.
Consumers tend to trust their wearable scores—should they? In this News and Perspectives op-ed, JMIR Correspondent Blythe Karow reports on the different types of accuracy in wearables and how wearable companies can help close the gap.
Richter and colleagues offer a careful systematic review of how large language model–based tools affect physician-patient communication, but the outcomes that matter most in deciding to scale a solution—trust and safety—are still unknown. Before deploying these tools, health systems need local validation, ongoing monitoring for accuracy and equity, and pragmatic trials that measure trust, safety, and outcomes alongside perceived empathy.
Background Clinical notes are rich sources of clinical information and are routinely used in clinical practice to infer past diagnoses and other comorbidities. Large language models (LLMs) may simplify the task by extracting this information from clinical notes. Objective The study aims to evaluate the performance of state-of-the-art LLMs on the task of labeling English clinical notes. Methods This institutional review board–approved retrospective study comprised 380 clinical notes obtained from 2 institutions in the United States. The notes were manually labeled for the presence or absence of 29 conditions (simplified Elixhauser Comorbidity Index) by 2 human readers—a radiology resident and a medical student under the supervision of a senior radiologist. Forty-six LLMs, including 4 closed-source commercial models from the GPT and Claude families, and 42 open-weights LLMs from the Llama (13 models) and the Qwen (29 models) families were used to label each note. The GPT-o3-mini-high reasoning model was used to parse the free-text model responses to binary yes/no answers. Using the manual labels as the reference standard, the recall, precision, and F 1 -scores for the models were evaluated. The effect of factors such as the length of the notes and the family, size, and release recency of the LLMs was analyzed using generalized linear models. Results The top-performing LLMs included the GPT and Claude models, and the larger open-weights models included Llama 3 70B, Llama 3.1 70B, Llama 3.1 405B, Llama 3.3 70B, Qwen 1.5 72B, Qwen 1.5 110B, Qwen 2 72B, and Qwen 2 VL 72B, which achieved F 1 -scores above 70%. The most recent and higher-capacity releases of the LLMs have been improving mainly in their precision. The evaluation of 29 conditions showed a large variance in performance depending on the condition. Conclusions In this evaluation with 46 LLMs, larger open-weights LLMs such as the Llama 3.3 70B performed on par with or better than the GPT and Claude models, achieving F 1 -scores above 70%. The performance of open-weights models improved significantly with model size and recency. These results suggest that locally run open-weights LLMs can be potentially used to label clinical notes.
Unlabelled Intravenous fluid therapy is a cornerstone of optimizing body hydration, circulation, and perfusion, yet its rational implementation is limited by inconsistent clinical terminology. As fluid therapy moves toward AI-empowered clinical decision support systems (AI-CDSS), there is a critical need for high-quality, verified training data. The reliability of these systems is currently threatened by misleading data resulting from ambiguous definitions and the interchangeable use of nonsynonymous terms, such as "dehydration" and "hypovolemia." This multidisciplinary panel-of-experts opinion article intends to start a discussion by identifying the key drawbacks in current terminology and proposes solutions, including a shift toward adequacy-based phenotypes and categorizing blood and water volumes as "insufficient," "optimal," or "excessive" rather than relying on context-sensitive quantitative measures such as "low," "normal," or "high." Illustrative scenarios are used to show how inconsistent terminology and static quantitative labels could mislead an AI-CDSS. The proposed phenotype matrix is designed to provide a structured, systematic description of fluid requirements for specific clinical objectives that can be integrated into clinical algorithms and machine learning databases. An example of a fluid requirement assessment algorithm is used for exploring its applicability in AI training. The proposed terminology and phenotype matrix are provisional: they represent the opinion of a multidisciplinary expert panel and have not undergone formal consensus, clinical validation, or testing within an AI system. Broad consensus-building initiatives will follow.
