OBJECTIVES:We aimed to develop a data model and a natural language processing (NLP) pipeline for representing physical activity (PA) in Electronic Health Records (EHRs), and to evaluate transformer- and Large Language Model (LLM)-based classifiers for sentence-level PA attribute classification. MATERIALS AND METHODS:We analyzed PA documentation across three patient cohorts (cancer, COVID, and Alzheimer's disease) using structured and unstructured EHR data. A conceptual schema was developed to represent PA and its linguistic attributes. Five BERT models and three modern LLMs (Llama3-8B, MedAlpaca-13B, and PMC-Llama-13B) were evaluated for classifying PA attributes (binary status, negation, exclusion, and an eleven-class Category) on pre-extracted PA-related sentences. RESULTS:Clinical notes were a richer source of PA information than structured ICD or SDoH data. On binary tasks, the best BERT model reached F1 0.619 (Exclusion); with Supervised Fine-Tuning (SFT), Llama3-8B reached F1 0.689 (Exclusion). On the 11-class Category task, performance was modest (best macro-F1 0.262, ROC-AUC 0.803, by Llama3-8B). DISCUSSION:In-Context Learning (ICL) was highly variable: while Llama3-8B-ICL achieved the best ROC-AUC on Category, the domain-specific MedAlpaca-13B and PMC-LLaMA-13B essentially failed. These results, together with sparsely represented PA elements (Amount, Frequency, Assessment) and the absence of a downstream evaluation, position this work as an initial proof of feasibility, with supervised domain adaptation still required for reliable clinical PA extraction. CONCLUSION:We contribute a PA data model, annotation schema, and a working NLP pipeline with a BERT/LLM benchmark for sentence-level PA attribute classification. The pipeline supports future end-to-end PA extraction and downstream applications such as phenotyping, risk prediction, and cohort identification.

Alzheimer’s disease (AD) and related dementias (ADRD) are common in older adults, their prevention and management are challenging problems. To prevent or delay ADRD, dietary supplements (DS) have emerged as a promising treatment; however, the role of DS usage on disease progression of patients with cognitive impairments remains unclear. Little clinical trial evidence is available, but substantial information is contained in electronic health records (EHR), including structured and unstructured data about patients’ DS usage and disease status. The objectives of this study were to (1) develop accurate natural language processing (NLP) methods to extract DS usage for patients with Mild Cognitive Impairment (MCI) and ADRD, (2) examine the coverage of DS in structured data versus unstructured data and (3) compare DS usage information in EHR with National Health and Nutrition Examination Survey (NHANES) data. We collected EHR data for patients with MCI and ADRD. A pipeline to extract the usage information of DS from both structured data and unstructured clinical notes was developed in the study. For structured data, we used the medication table to identify the DS and for unstructured clinical notes, we applied Bidirectional Encoder Representations from Transformers (BERT) fine-tuning strategy to extract the DS usage status. The best named entity recognition model for DS achieved an F1-score of 0.964 and the PubMed BERT-based use status classifier had a weighted F1-score of 0.879. We applied these models to extract DS usage information from unstructured clinical notes and subsequently compared and combined with those from structured medication orders. In total, 125 unique DS were identified for patients with MCI and 108 unique DS were identified for patients with ADRD. In this study, we developed an NLP-based pipeline to extract the DS use information from medication structured data and clinical notes in EHR for patients with MCI and ADRD. Our method could further help understand the DS usage of patients with MCI and ADRD, and how these DS could influence the diseases.

Background Error analysis plays a crucial role in clinical concept extraction, a fundamental subtask within clinical natural language processing (NLP). The process typically involves a manual review of error types, such as contextual and linguistic factors contributing to their occurrence, and the identification of underlying causes to refine the NLP model and improve its performance. Conducting error analysis can be complex, requiring a combination of NLP expertise and domain-specific knowledge. Due to the high heterogeneity of electronic health record (EHR) settings across different institutions, challenges may arise when attempting to standardize and reproduce the error analysis process.Objectives This study aims to facilitate a collaborative effort to establish common definitions and taxonomies for capturing diverse error types, fostering community consensus on error analysis for clinical concept extraction tasks.Materials and Methods We iteratively developed and evaluated an error taxonomy based on existing literature, standards, real-world data, multisite case evaluations, and community feedback. The finalized taxonomy was released in both .dtd and .owl formats at the Open Health Natural Language Processing Consortium. The taxonomy is compatible with several different open-source annotation tools, including MAE, Brat, and MedTator.Results The resulting error taxonomy comprises 43 distinct error classes, organized into 6 error dimensions and 4 properties, including model type (symbolic and statistical machine learning), evaluation subject (model and human), evaluation level (patient, document, sentence, and concept), and annotation examples. Internal and external evaluations revealed strong variations in error types across methodological approaches, tasks, and EHR settings. Key points emerged from community feedback, including the need to enhancing clarity, generalizability, and usability of the taxonomy, along with dissemination strategies.Conclusion The proposed taxonomy can facilitate the acceleration and standardization of the error analysis process in multi-site settings, thus improving the provenance, interpretability, and portability of NLP models. Future researchers could explore the potential direction of developing automated or semi-automated methods to assist in the classification and standardization of error analysis.

Language models (LMs) such as BERT and GPT have revolutionized natural language processing (NLP). However, the medical field faces challenges in training LMs due to limited data access and privacy constraints imposed by regulations like the Health Insurance Portability and Accountability Act (HIPPA) and the General Data Protection Regulation (GDPR). Federated learning (FL) offers a decentralized solution that enables collaborative learning while ensuring data privacy. In this study, we evaluated FL on 2 biomedical NLP tasks encompassing 8 corpora using 6 LMs. Our results show that: (1) FL models consistently outperformed models trained on individual clients’ data and sometimes performed comparably with models trained with polled data; (2) with the fixed number of total data, FL models training with more clients produced inferior performance but pre-trained transformer-based models exhibited great resilience. (3) FL models significantly outperformed pre-trained LLMs with few-shot prompting.

Objective This study leverages the rich diversity of the All of Us Research Program (All of Us)'s dataset to devise a predictive model for cardiovascular disease (CVD) in breast cancer (BC) survivors. Central to this endeavor is the creation of a robust data integration pipeline that synthesizes electronic health records (EHRs), patient surveys, and genomic data, while upholding fairness across demographic variables.Materials and Methods We have developed a universal data wrangling pipeline to process and merge heterogeneous data sources of the All of Us dataset, address missingness and variance in data, and align disparate data modalities into a coherent framework for analysis. Utilizing a composite feature set including EHR, lifestyle, and social determinants of health (SDoH) data, we then employed Adaptive Lasso and Random Forest regression models to predict 6 CVD outcomes. The models were evaluated using the c-index and time-dependent Area Under the Receiver Operating Characteristic Curve over a 10-year period.Results The Adaptive Lasso model showed consistent performance across most CVD outcomes, while the Random Forest model excelled particularly in predicting outcomes like transient ischemic attack when incorporating the full multi-model feature set. Feature importance analysis revealed age and previous coronary events as dominant predictors across CVD outcomes, with SDoH clustering labels highlighting the nuanced impact of social factors.Discussion The development of both Cox-based predictive model and Random Forest Regression model represents the extensive application of the All of Us, in integrating EHR and patient surveys to enhance precision medicine. And the inclusion of SDoH clustering labels revealed the significant impact of sociobehavioral factors on patient outcomes, emphasizing the importance of comprehensive health determinants in predictive models. Despite these advancements, limitations include the exclusion of genetic data, broad categorization of CVD conditions, and the need for fairness analyses to ensure equitable model performance across diverse populations. Future work should refine clinical and social variable measurements, incorporate advanced imputation techniques, and explore additional predictive algorithms to enhance model precision and fairness.Conclusion This study demonstrates the liability of the All of Us's diverse dataset in developing a multi-modality predictive model for CVD in BC survivors risk stratification in oncological survivorship. The data integration pipeline and subsequent predictive models establish a methodological foundation for future research into personalized healthcare.

Accurately predicting heart disease risks in patients with breast cancer is crucial for clinical decision support and patient safety. This study developed and evaluated predictive models for six heart diseases using real-world electronic health records (EHRs) data. We incorporated a trainable decay mechanism to handle missing values in the long short-term memory (LSTM) model, creating LSTM-D models to predict heart disease risk based on longitudinal EHRs data. Additionally, we deployed NLP methods to extract breast cancer phenotypes from clinical texts, integrating unstructured and structured data to enhance predictions. Our LSTM-D models outperformed baseline models in predicting congestive heart failure, coronary artery disease, cardiomyopathy, myocardial infarction, transient ischemic attack, and aortic regurgitation, with AUC scores ranging from 0.7189 to 0.9548. Observation windows of 12-24 months were found optimal for model performance. This research advances precise, personalized care strategies, enabling early intervention and improved management of cardiovascular risks in breast cancer survivors.

Reinforcement learning (RL) has emerged as a powerful approach for tackling complex medical decision-making problems such as treatment planning, personalized medicine, and optimizing the scheduling of surgeries and appointments. It has gained significant attention in the field of Natural Language Processing (NLP) due to its ability to learn optimal strategies for tasks such as dialogue systems, machine translation, and question-answering. This paper presents a review of the RL techniques in NLP, highlighting key advancements, challenges, and applications in healthcare. The review begins by visualizing a roadmap of machine learning and its applications in healthcare. And then it explores the integration of RL with NLP tasks. We examined dialogue systems where RL enables the learning of conversational strategies, RL-based machine translation models, question-answering systems, text summarization, and information extraction. Additionally, ethical considerations and biases in RL-NLP systems are addressed.

Language models (LMs) such as BERT and GPT have revolutionized natural language processing (NLP). However, the medical field faces challenges in training LMs due to limited data access and privacy constraints imposed by regulations like the Health Insurance Portability and Accountability Act (HIPPA) and the General Data Protection Regulation (GDPR). Federated learning (FL) offers a decentralized solution that enables collaborative learning while ensuring data privacy. In this study, we evaluated FL on 2 biomedical NLP tasks encompassing 8 corpora using 6 LMs. Our results show that: 1) FL models consistently outperformed models trained on individual clients' data and sometimes performed comparably with models trained with polled data; 2) with the fixed number of total data, FL models training with more clients produced inferior performance but pre-trained transformer-based models exhibited great resilience. 3) FL models significantly outperformed large language models using zero-/one-shot learning and offered lightning inference speed.

Objective:Transformer-based language models are prevailing in the clinical domain due to their excellent performance on clinical NLP tasks. The generalizability of those models is usually ignored during the model development process. This study evaluated the generalizability of CancerBERT, a Transformer-based clinical NLP model, along with classic machine learning models, i.e., conditional random field (CRF), bi-directional long short-term memory CRF (BiLSTM-CRF), across different clinical institutes through a breast cancer phenotype extraction task. Materials and methods:Two clinical corpora of breast cancer patients were collected from the electronic health records from the University of Minnesota (UMN) and Mayo Clinic (MC), and annotated following the same guideline. We developed three types of NLP models (i.e., CRF, BiLSTM-CRF and CancerBERT) to extract cancer phenotypes from clinical texts. We evaluated the generalizability of models on different test sets with different learning strategies (model transfer vs locally trained). The entity coverage score was assessed with their association with the model performances. Results:We manually annotated 200 and 161 clinical documents at UMN and MC. The corpora of the two institutes were found to have higher similarity between the target entities than the overall corpora. The CancerBERT models obtained the best performances among the independent test sets from two clinical institutes and the permutation test set. The CancerBERT model developed in one institute and further fine-tuned in another institute achieved reasonable performance compared to the model developed on local data (micro-F1: 0.925 vs 0.932). Conclusions:The results indicate the CancerBERT model has superior learning ability and generalizability among the three types of clinical NLP models for our named entity recognition task. It has the advantage to recognize complex entities, e.g., entities with different labels.

Remote patient monitoring (RPM) programs are being increasingly utilized in the care of patients to manage acute and chronic disease including with acute COVID-19. The goal of this study is to explore the topics and patterns of patients' messages to the care team in an RPM program in patients with presumed COVID-19. We conducted a topic analysis to 6,262 comments from 3,248 patients enrolled in the COVID-19 RMP at M Health Fairview. Evaluation of comments was performed using LDA and CorEx topic modeling. Subject matter experts evaluated topic models, including identification of and defining topics and categories. Topics plotted over time to identify trends in topic weights over the enrollment period. The overall accuracy of comments assignment to topics by LDA and CorEx models were 72.8% and 88.2%. Most identified topics focused on signs and symptoms of COVID-19. Topics related to COVID-19 diagnosis demonstrated a correlation with announcements of availability of viral and antibody testing in national and local media.

Language models (LMs) like BERT and GPT have revolutionized natural language processing (NLP). However, privacy-sensitive domains, particularly the medical field, face challenges to train LMs due to limited data access and privacy constraints imposed by regulations like the Health Insurance Portability and Accountability Act (HIPPA) and the General Data Protection Regulation (GDPR). Federated learning (FL) offers a decentralized solution that enables collaborative learning while ensuring the preservation of data privacy. In this study, we systematically evaluate FL in medicine across 2 biomedical NLP tasks using 6 LMs encompassing 8 corpora. Our results showed that: 1) FL models consistently outperform LMs trained on individual client’s data and sometimes match the model trained with polled data; 2) With the fixed number of total data, LMs trained using FL with more clients exhibit inferior performance, but pre-trained transformer-based models exhibited greater resilience. 3) LMs trained using FL perform nearly on par with the model trained with pooled data

Abstract Objective Accurate extraction of breast cancer patients’ phenotypes is important for clinical decision support and clinical research. This study developed and evaluated cancer domain pretrained CancerBERT models for extracting breast cancer phenotypes from clinical texts. We also investigated the effect of customized cancer-related vocabulary on the performance of CancerBERT models. Materials and Methods A cancer-related corpus of breast cancer patients was extracted from the electronic health records of a local hospital. We annotated named entities in 200 pathology reports and 50 clinical notes for 8 cancer phenotypes for fine-tuning and evaluation. We kept pretraining the BlueBERT model on the cancer corpus with expanded vocabularies (using both term frequency-based and manually reviewed methods) to obtain CancerBERT models. The CancerBERT models were evaluated and compared with other baseline models on the cancer phenotype extraction task. Results All CancerBERT models outperformed all other models on the cancer phenotyping NER task. Both CancerBERT models with customized vocabularies outperformed the CancerBERT with the original BERT vocabulary. The CancerBERT model with manually reviewed customized vocabulary achieved the best performance with macro F1 scores equal to 0.876 (95% CI, 0.873–0.879) and 0.904 (95% CI, 0.902–0.906) for exact match and lenient match, respectively. Conclusions The CancerBERT models were developed to extract the cancer phenotypes in clinical notes and pathology reports. The results validated that using customized vocabulary may further improve the performances of domain specific BERT models in clinical NLP tasks. The CancerBERT models developed in the study would further help clinical decision support.

Alzheimer's Disease (AD) is a progressive brain disorder that causes declined cognition and affects the patients' daily activities. It is the most common cause of dementia in people older than 65 years in America [1]. About 6.2 million people aged 65 and older were diagnosed with Alzheimer's disease and Alzheimer's disease-related dementias (AD/ADRD) in the U.S. in 2021, and the number is projected to increase to 7.2 million in 2025 and to 13.8 million in 2060 [1]. Health disparities are characterized as preventable gaps in the burden of the disease across different races, socio-economic groups, sex or other subgroups. It is noticed that the prevalence of AD/ADRD is higher among African Americans than Whites, which indicates that racial/ethnic disparity may exist in this disease [2]–[4]. There are no efficient treatments to prevent or reverse the progression of this disease. Unlike AD/ADRD patients, people with mild cognitive impairment (MCI), a prodromal stage of AD/ADRD, can still perform daily activities independently, which makes studies about the progression from MCI to AD/ADRD important. However, limited studies have investigated the potential existence of racial disparities from the disease progression perspective, to the best of our knowledge [5]. This study aimed to fill this gap by evaluating the racial disparities in the progression from MCI to AD/ADRD among African Americans and Whites and examining the promising predictors of disease progression for the overall group and two racial groups separately.

Cardiotoxicity induced by the breast cancer treatments (i.e., chemotherapy, targeted therapy and radiation therapy) is a significant problem for breast cancer patients. The cardiotoxicity risk for breast cancer patients receiving different treatments remains unclear. We developed and evaluated risk predictive models for cardiotoxicity in breast cancer patients using EHR data. The AUC scores to predict the CHF, CAD, CM and MI are 0.846, 0.857, 0.858 and 0.804 respectively. After adjusting for baseline differences in cardiovascular health, patients who received chemotherapy or targeted therapy appeared to have higher risk of cardiotoxicity than patients who received radiation therapy. Due to differences in baseline cardiac health across the different breast cancer treatment groups, caution is recommended in interpreting the cardiotoxic effect of these treatments.

The roles of dietary supplement (DS) usage on disease progression of patients with cognitive impairments remain unclear. Transformed-based language models were trained to identify DS use status from clinical notes among patients with Alzheimer’s disease and related dementias (ADRD). The best name entity recognition for DS achieved F1-score is 0.964 and the PubMed BERT based use status classifier achieved the weighted F1-score of 0.879. Integrating with DS use from medication table, we identified totally 125 unique DS among patients with mild cognitive impairment (MCI) only and 108 unique DS among patients who progressed to ADRD.

Accurate extraction of breast cancer patients' phenotypes is important for clinical decision support and clinical research. Current models do not take full advantage of cancer domain-specific corpus, whether pre-training Bidirectional Encoder Representations from Transformer model on cancer-specific corpus could improve the performances of extracting breast cancer phenotypes from texts data remains to be explored. The objective of this study is to develop and evaluate the CancerBERT model for extracting breast cancer phenotypes from clinical texts in electronic health records. This data used in the study included 21,291 breast cancer patients diagnosed from 2010 to 2020, patients' clinical notes and pathology reports were collected from the University of Minnesota Clinical Data Repository (UMN). Results: About 3 million clinical notes and pathology reports in electronic health records for 21,291 breast cancer patients were collected to train the CancerBERT model. 200 pathology reports and 50 clinical notes of breast cancer patients that contain 9,685 sentences and 221,356 tokens were manually annotated by two annotators. 20% of the annotated data was used as a test set. Our CancerBERT model achieved the best performance with macro F1 scores equal to 0.876 (95% CI, 0.896-0.902) for exact match and 0.904 (95% CI, 0.896-0.902) for the lenient match. The NER models we developed would facilitate the automated information extraction from clinical texts to further help clinical decision support. Conclusions and Relevance: In this study, we focused on the breast cancer-related concepts extraction from EHR data and obtained a comprehensive annotated dataset that contains 7 types of breast cancer-related concepts. The CancerBERT model with customized vocabulary could significantly improve the performance for extracting breast cancer phenotypes from clinical texts.

OBJECTIVE:We sought to demonstrate the feasibility of utilizing deep learning models to extract safety signals related to the use of dietary supplements (DSs) in clinical text.MATERIALS AND METHODS:Two tasks were performed in this study. For the named entity recognition (NER) task, Bi-LSTM-CRF (bidirectional long short-term memory conditional random field) and BERT (bidirectional encoder representations from transformers) models were trained and compared with CRF model as a baseline to recognize the named entities of DSs and events from clinical notes. In the relation extraction (RE) task, 2 deep learning models, including attention-based Bi-LSTM and convolutional neural network as well as a random forest model were trained to extract the relations between DSs and events, which were categorized into 3 classes: positive (ie, indication), negative (ie, adverse events), and not related. The best performed NER and RE models were further applied on clinical notes mentioning 88 DSs for discovering DSs adverse events and indications, which were compared with a DS knowledge base.RESULTS:For the NER task, deep learning models achieved a better performance than CRF, with F1 scores above 0.860. The attention-based Bi-LSTM model performed the best in the RE task, with an F1 score of 0.893. When comparing DS event pairs generated by the deep learning models with the knowledge base for DSs and event, we found both known and unknown pairs.CONCLUSIONS:Deep learning models can detect adverse events and indication of DSs in clinical notes, which hold great potential for monitoring the safety of DS use.

Abstract Objective Dietary supplements are widely used. However, dietary supplements are not always safe. For example, an estimated 23 000 emergency room visits every year in the United States were attributed to adverse events related to dietary supplement use. With the rapid development of the Internet, consumers usually seek health information including dietary supplement information online. To help consumers access quality online dietary supplement information, we have identified trustworthy dietary supplement information sources and built an evidence–based knowledge base of dietary supplement information—the integrated DIetary Supplement Knowledge base (iDISK) that integrates and standardizes dietary supplement related information across these different sources. However, as information in iDISK was collected from scientific sources, the complex medical jargon is a barrier for consumers’ comprehension. The objective of this study is to assess how different approaches to simplify and represent dietary supplement information from iDISK will affect lay consumers’ comprehension. Materials and Methods Using a crowdsourcing platform, we recruited participants to read dietary supplement information in 4 different representations from iDISK: (1) original text, (2) syntactic and lexical text simplification (TS), (3) manual TS, and (4) a graph–based visualization. We then assessed how the different simplification and representation strategies affected consumers’ comprehension of dietary supplement information in terms of accuracy and response time to a set of comprehension questions. Results With responses from 690 qualified participants, our experiments confirmed that the manual approach, as expected, had the best performance for both accuracy and response time to the comprehension questions, while the graph–based approach ranked the second outperforming other representations. In some cases, the graph–based representation outperformed the manual approach in terms of response time. Conclusions A hybrid approach that combines text and graph–based representations might be needed to accommodate consumers’ different information needs and information seeking behavior.

Natural language processing (NLP) is a subfield of artificial intelligence that is concerned with the automatic understanding of human language by computers. NLP has seen much success in recent years due to increased computing power and the rise of deep learning, and this success has extended into the domains of biomedical and clinical text. NLP has contributed to tasks such as the discovery of drug interactions, the development of clinical decision support systems, and the facilitation of chart review. Part I of this chapter provides an introduction to NLP, some common tasks in the biomedical domain, and the methods for accomplishing these tasks. Part II gives a survey of recent applications of NLP in cardiovascular medicine.