Remote patient monitoring (RPM) has shown potential benefits for patients with chronic conditions. This study aimed to compare healthcare utilization between RPM and non-RPM patients with congestive heart failure (CHF) or chronic obstructive pulmonary disease (COPD). RPM patients were identified based on a history of CHF and COPD and receipt of RPM services. A 1:4 propensity score matching approach was used to identify comparable non-RPM patients with CHF or COPD, adjusting for demographics, comorbidity burden, and baseline healthcare use. The final analytic cohort comprised 1,050 patients. RPM patients were generally older adults with high Charlson Comorbidity Index scores. Compared to non-RPM patients, those receiving RPM had a significantly shorter hospital length of stay (2.4±5.4 vs. 4.7±11.3 days, p<0.01) and reduced frequency of hospitalizations (0.6±1.1 vs. 1.7±3.9, p<0.01) on an annual basis. Our results demonstrated the utility of RPM in the management of patients with chronic cardiopulmonary conditions.
Even a short delay in bladder cancer diagnosis and treatment are associated with significantly worse outcomes. Early identification of high-risk individuals may enable timely evaluation and improve prognosis. This study aimed to develop predictive models for early bladder cancer diagnoses using structured claims data. We used the MarketScan® database and extracted information from three relational tables: enrollment, outpatient diagnoses and procedures, and outpatient medications. Bladder cancer cases were identified using ICD-9 code 188 and ICD-10 code C67, restricted to first-time diagnoses between 2016 and 2019. Patients aged 50 years or older with at least two consecutive years of continuous enrollment were included. A control group of non-cancer patients was selected using propensity score matching on age, sex, Charlson Comorbidity Index, and diagnosis year. For machine learning, we developed three XGBoost models to predict bladder cancer 1, 2, and 3 months prior to diagnosis, using a one-year activity window. Diagnoses, procedures, and medications were grouped into clinically meaningful categories. A two-stage feature selection process was applied to reduce dimensionality while maintaining interpretability. The final dataset included 5,469 bladder cancer patients and 5,469 matched controls. The models achieved an AUC of 0.81 for 1-month-ahead prediction and 0.71 for both 2- and 3-months-ahead predictions. Feature importance analysis revealed consistent predictive signals across models, with hematuria as the most dominant feature. Procedures such as endoscopy and CT imaging were also strongly associated with future bladder cancer diagnosis.
Delirium is a common ICU complication associated with increased morbidity, mortality, and healthcare costs. Early identification of at-risk patients can enable preventive interventions. In this study, we developed machine learning models to predict delirium onset using clinical data from the first 24 hours of ICU admission, extracted from the MIMIC-IV database. Adult patients with ICU stays longer than 24 hours and no delirium on day 1 were included. Delirium was defined as a positive CAM-ICU assessment between ICU days 2 and 7 in patients with RASS ≥ 3. Forty-two first-day clinical features were used to train SVM, random forest, and XGBoost models. Random forest and XGBoost achieved AUCs of 0.749 and 0.757, with first-day GCS and RASS identified as the most important predictors, highlighting the value of early ICU data for delirium risk prediction.
Background:Accurate identification of clinical symptoms and signs (S&S) is essential for the early detection of high-burden cardiorespiratory conditions, including lung cancer, chronic obstructive pulmonary disease, and heart failure. Although symptom data play a central role in diagnostic reasoning and predictive modeling, most S&S information remains embedded in unstructured electronic health record notes, limiting their use in automated phenotyping, surveillance, and clinical decision support. Traditional natural language processing systems struggle with domain variability and contextual nuance in clinical text. Recent advances in large language models (LLMs) offer a promising alternative, yet challenges remain in hallucinations, overinference, and safe deployment. This study evaluated whether locally deployed open-source models could reliably extract cardiorespiratory S&S and map them to ICD-10-CM (International Classification of Diseases, Tenth Revision, Clinical Modification) codes using optimized prompting strategies. Objective:This study aims to assess the accuracy of open-source LLMs in extracting explicitly stated cardiorespiratory S&S from clinical notes and mapping them to ICD-10-CM codes (R00-R09) and to compare performance across 4 prompt-engineering strategies, including a multimodule LLM framework. Methods:A total of 593 clinical notes from the MTSamples database were manually reviewed, with 93 notes used for prompt development and comparison using Llama 3.3-70B, and 500 notes used as testing data for the final best prompt setting using both Llama 3.3-70B and gpt-oss-120B. Four prompting conditions were evaluated: (1) instruction-only, (2) ICD-10-CM definition-based prompts, (3) assumption-free prompts, and (4) a multimodule LLM framework with postprocessing. Performance was measured using precision, recall, and F1-score for both S&S extraction and ICD-10-CM code generation. Results:Across all prompt strategies, model performance improved as more structure and constraints were added. Instruction-only prompting demonstrated high recall but poor precision. Incorporating ICD-10-CM definitions improved coding accuracy, and assumption-free prompting further balanced precision and recall. The multimodule approach with postprocessing achieved the highest performance during prompt development. On the independent test corpus, entity-level microaveraged evaluation showed that gpt-oss-120B outperformed Llama 3.3-70B in both tasks. For S&S extraction, Llama 3.3-70B achieved a precision of 0.63, a recall of 0.86, and an F1-score of 0.73, whereas gpt-oss-120B achieved a precision of 0.89, a recall of 0.87, and an F1-score of 0.88. For ICD-10-CM code mapping, Llama 3.3-70B achieved a precision of 0.59, a recall of 0.83, and an F1-score of 0.69, whereas gpt-oss-120B achieved a precision of 0.90, a recall of 0.84, and an F1-score of 0.87. Conclusions:Locally deployed LLMs, when paired with optimized prompting and multimodule orchestration, can accurately extract cardiorespiratory S&S and generate ICD-10-CM codes from unstructured clinical notes. This approach increases the level of data safety by enabling on-premises processing without external data transmission and demonstrates strong potential for scalable, domain-adaptive symptom extraction pipelines in biomedical informatics. Future work should expand datasets and evaluate generalizability across clinical domains.
Aim:This study aimed to evaluate the performance of machine learning (ML) models in predicting patient no-shows for telemedicine appointments within Peruvian health system and identify key predictors of nonattendance. Methods:We performed a retrospective observational study using anonymized data (June 2019-November 2023) from "Teleatiendo." The dataset included over 1.5 million completed appointments and about 64,000 no-shows (4.1%), focusing on teleorientation and telemonitoring. Predictor variables included patient demographics, socioeconomic factors, health care facility characteristics, appointment timing, and telemedicine service types. A 70% training, 10% validation, and 20% testing split were used over 10 iterations, with hyperparameter tuning performed on the validation set to identify optimal model parameters. Multiple ML approaches-random forest, XGBoost, LightGBM, and anomaly detection-were implemented in combination with undersampling and cost-sensitive learning to address class imbalance. Performance was evaluated using precision, recall, specificity, area under the curve (AUC), F1-score, and accuracy. Results:Of the models tested, undersampling with XGBoost achieved a precision of 0.115 (±0.001), recall of 0.654 (±0.005), specificity of 0.786 (±0.002), AUC of 0.720 (±0.002), and accuracy of 0.780 (±0.002). In contrast, cost-sensitive XGBoost exhibited a balanced performance with a precision of 0.123 (±0.001), recall of 0.639 (±0.006), specificity of 0.805 (±0.004), AUC of 0.722 (±0.001), and accuracy of 0.799 (±0.003). Additionally, cost-sensitive random forest achieved the highest specificity (0.843 ± 0.002) and accuracy (0.832 ± 0.001) but recorded a lower recall (0.585 ± 0.004), while cost-sensitive LightGBM and balanced random forest yielded performance metrics similar to cost-sensitive XGBoost. Isolation forest, used for abnormality detection, demonstrated the lowest performance. Conclusions:ML models can moderately predict telemedicine no-shows in Peru, with cost-sensitive boosting techniques enhancing the identification of high-risk patients. Key predictors reflect both individual behavior and system-level contexts, suggesting the need for tailored, context-specific interventions. These findings can inform targeted strategies to optimize telemedicine, improve appointment adherence, and promote equitable health care access.
This study examined diagnostic discrepancies between emergency department (ED) and inpatient care using three distinct datasets: the All of Us Research Program, a synthetic VA dataset, and TriNetX. Inclusion criteria focused on patients admitted through the ED, with diagnoses compared at two levels: category (530+ categories) and body system (22 categories). A many-to-many comparison was used for the All of Us and TriNetX datasets, while a one-to-one comparison was used for the VA dataset due to its limited diagnosis records. The analytical datasets comprised patients with chronic cardiopulmonary conditions, with All of Us containing 599 patients, TriNetX 1,562 patients, and the VA dataset 12,621 patients. High mismatch rates were consistently observed. At the category level, no matches were seen in 19.10% of All of Us cases, 20.30% of TriNetX cases, and 58.30% of VA cases. For the body system level, no matches occurred in 7.78%, 7.70%, and 30.30% of cases, respectively. Partial matches were more common in patients with high comorbidities and inaccurate diagnoses were associated with longer hospital stays.
Accurate and timely diagnoses are essential in emergency medicine, where delays in care can have severe consequences. The goal of the study was to use the MIMIC-III dataset to evaluate ClinicalBERT and GPT-4 models in predicting discharge diagnoses using inpatient notes. In GPT-4 diagnostic prediction, we investigated two prompt engineering methods: the binary voting method and the scaled likelihood rating method. We tested four conditions: circulatory system diseases, respiratory diseases, septicemia, and pneumonia, with 160 cases analyzed. ClinicalBERT outperformed GPT models, achieving the highest F1 scores across all categories. It scored 0.816 for circulatory diseases, 0.952 for respiratory diseases, 0.889 for septicemia, and 0.743 for pneumonia. The GPT binary voting method performed slightly better than the scaled likelihood rating method in most cases. While ClinicalBERT required significant computational resources, GPT-4 needed no pre-training, had high recall but lower precision, and could process longer text.
Accurately identifying patient signs and symptoms from clinical notes is essential for effective diagnosis, treatment planning, and medical research. In this study, we evaluated the performance of the Meta Llama model in extracting signs and symptoms related to the genitourinary system, along with their corresponding ICD-10 codes, from urological clinical notes in the MTSamples dataset. The dataset was manually annotated to compare the extraction results of large language models (LLMs) output. We utilized Llama 3.3-70B and performed prompt engineering. The findings suggest that the best performance was achieved when the prompt included a predefined list of definitions of corresponding ICD-10 codes and restricted the model from making assumptions. Under these conditions, Llama 3.3-70B achieved an average recall of 0.96, precision of 0.89, and F1-score of 0.92 for S&S extraction, as well as an average recall of 0.93, precision of 0.85, and F1-score of 0.89 for ICD-10 code generation.
In this study, we examined how well the open-source foundational large language models (LLMs) can extract symptoms and signs (S&S), along with their corresponding ICD-10 codes, from clinical notes found in the public MTSamples dataset. The dataset comprising notes of patients with genitourinary conditions was manually annotated to compare the S&S extraction results with outputs generated by LLMs. We assessed three versions of the Llama model-Llama 3.1-13B, Llama 3.3-70B, and Me-Llama-13B-focusing on their consistency, runtime, and performance. Each model was tested on two tasks: (1) S&S extraction and (2) ICD-10 code generation. Our findings indicate that Llama 3.3-70B performed the best overall. With fast runtime and high consistency, it achieved an average recall of 0.87 and an average precision of 0.71 for S&S extraction, as well as an average recall of 0.71 and an average precision of 0.54 for ICD-10 code generation.
This study explored the potential of LLMs, such as ClinicalBERT and GPT-4, to identify potential diagnoses using early clinical notes from the MIMIC-III dataset. We compared these models across four conditions: circulatory system diseases, respiratory system diseases, septicemia, and pneumonia. ClinicalBERT consistently outperformed the GPT models, with its highest F1-score of 0.952 for respiratory system diseases. The GPT models, while showing high recall, had lower precision, with the highest F1-score of 0.784 achieved by the GPT binary voting method. ClinicalBERT demonstrated strong precision and F1-scores, while GPT-4 excelled in recall.
This study investigated healthcare utilization patterns prior to prostate cancer diagnoses, aiming to develop machine learning models for early prediction of cancer diagnosis. Data from the All of Us Research Program was used, focusing on adult patients diagnosed with prostate cancer between 2010 and 2019. Key variables were derived from procedure, measurements, and condition records, including PSA values, comorbidity index, and symptoms. Multiple machine learning models were tested to predict prostate cancer 3, 6, 9, and 12 months ahead of time. The dataset included 1,276 cancer patients and 1,232 non-cancer patients. The XGBoost model performed best at 3 months, achieving an accuracy and F1 score of 0.73 and an AUC of 0.82. At 6 months, the model had an accuracy and F1 score of 0.71 and an AUC of 0.78. Performance declined with longer prediction windows. PSA values were consistently the most important predictor across all timeframes, along with other factors like triglyceride and creatinine levels.
This research aimed to develop a model for real-time prediction of aerobic exercise exertion levels. ECG signals were registered during 16-minute cycling exercises. Perceived ratings of exertion (RPE) were collected each minute from the study participants. Based on the reported RPE, each consecutive minute of the exercise was assigned to the "high exertion" or "low exertion" class. The characteristics of heart rate variability (HRV) in time and frequency domains were used as predictive features. The top ten ranked predictive features were selected using the minimum redundancy maximum relevance (mRMR) algorithm. The support vector machine demonstrated the highest accuracy with an F1 score of 82%.
The study used ClinicalBERT to predict body system categories based on clinical notes from the first three days of admission, using the MIMIC-III dataset. After data preprocessing, including the extraction of admission details, clinical notes, and diagnoses, the dataset comprised 510,956 notes associated with 44,270 unique hospitalizations. Discharge diagnoses were categorized into body systems, and the ClinicalBERT model was fine-tuned to predict associations with these diagnoses, resulting in 19 classification models-one for each body system. Around 80% of the models achieved F1 scores exceeding 0.7. Models for diseases of the circulatory, infectious and parasitic, respiratory, nervous, digestive, and genitourinary systems had F1 scores surpassing 0.8. Conversely, models for congenital malformations, eye and adnexa diseases, and ear and mastoid process diseases showed notably lower F1 scores. To explore model robustness, a comparison between three days and one day of notes per patient was conducted. While F1 scores generally decreased, a significant finding was that most body system models maintained satisfactory performance due to the statistical distribution similarities in note types and lengths between one and three days. This suggests the potential for ClinicalBERT's adaptability to varied data availability scenarios. Future studies could delve into developing a multiple notes model, testing its flexibility and robustness across different prediction durations, thereby potentially reducing the time and effort associated with model implementation in diverse clinical settings.
The goal of this study was to analyze diagnostic discrepancies between emergency department (ED) and hospital discharge diagnoses in patients with congestive heart failure admitted to the ED. Using a synthetic dataset from the Department of Veterans Affairs, the patients' primary diagnoses were compared at two levels: diagnostic category and body system. With 12,621 patients and 24,235 admission cases, the study found a 58% mismatch rate at the category level, which was reduced to 30% at the body system level. Diagnostic categories associated with higher levels of mismatch included aplastic anemia, pneumonia, and bacterial infections. In contrast, diagnostic categories associated with lower levels of mismatch included alcohol-related disorders, COVID-19, cardiac dysrhythmias, and gastrointestinal hemorrhage. Further investigation revealed that diagnostic mismatches are associated with longer hospital stays and higher mortality rates. These findings highlight the importance of reducing diagnostic uncertainty, particularly in specific diagnostic categories and body systems, to improve patient care following ED admission.
Diagnostic errors in emergency departments affect approximately 7.4 million patients every year. To address this, the integration of artificial intelligence, specifically Large Language Models (LLMs) like ClinicalBERT, into differential diagnostic process has been explored to reduce the diagnostic uncertainty and alleviate physician workload. This study focused on assessing the variance in diagnostic accuracy of LLMs between young and middle-aged adults (18-64 years) and older adults (65+ years) using 13 models fine-tuned on the MIMIC-III dataset, each targeting a specific body system. There were 8,321 cases of hospital stays and 124,736 clinical notes in the analytic dataset. Results indicated that while some models performed consistently well across both age groups, there was a discernible variability in others. 77% of models showed strong performance for younger adults, compared to 54% for older adults. The neoplasm (NEO) and circulatory (CIR) models stood out in both groups. In addition, the respiratory systems showed better performance for older adults. The mental and behavioral (MBD) system models, however, demonstrated a significant decline in recall for older adults. These findings highlight the importance of incorporating age-specific adjustments into AI models to optimize diagnostic precision across diverse patient populations.
Hypercapnic respiratory failure (an accumulation of carbon dioxide, CO2, in the blood) is often missed in clinical practice. Arterial blood gas is the standard diagnostic test, but it is painful and not routine. When clinicians fail to make the diagnosis, it is often because an arterial blood gas was not obtained. This 'partial verification' of CO2 levels presents a challenge for machine learning algorithms. We assessed the accuracy of two machine learning methods using demographics and routine lab work to estimate the likelihood that a patient has hypercapnic respiratory failure at hospital admission. Hospitalized patients who received an arterial blood gas sample constituted the training (n = 111,015) and geographic validation (n = 20,834) sets. Acceptance of "silver standard" diagnostic criteria and weighting observations by their modeled likelihood of receiving arterial blood gas sampling were used to assess the stability of findings in the presence of partial verification. Both regularized logistic regression and randomforest-based models resulted in acceptable performance (area under the curve: 0.763 and 0.758 respectively), with minimal changes in the auxiliary analyses. This work suggests that routinely available health record data can stratify the likelihood of hypercapnic respiratory failure among hospitalized adults, and findings may generalize to patients who have not received arterial blood gas sampling in clinical practice.
Multiple myeloma (MM) is one of the most common hematological malignancies. The goal of this study was to analyze the sociodemographic, economic, and genetic characteristics of long-term and short-term survival of multiple myeloma patients using EHR data from an academic medical center in New York City. The de-identified analytical dataset comprised 2,111 patients with MM who were stratified based on the length of survival into two groups. Demographic variables, cancer stage, income level, and genetic mutations were analyzed using descriptive statistics and logistic regression. Age, race, and cancer stage were all significant factors that affected the length of survival of multiple myeloma patients. In contrast, gender and income level were not significant factors based on the multivariate adjusted analysis. Older adults, African American patients, and patients who were diagnosed with stage III of multiple myeloma were the people most likely to exhibit short-term survival after the MM diagnosis.
Purpose This paper focuses on developing and testing three versions of interactive bike (iBikE) interfaces for remote monitoring and control of cycling exercise sessions to promote upper and lower limb rehabilitation. Methods Two versions of the system, which consisted of a portable bike and a tablet PC, were designed to communicate through either Bluetooth low energy (BLE) or Wi-Fi interfaces for real-time monitoring of exercise progress by both the users and their clinical team. The third version of the iBikE system consisted of a motorized bike and a tablet PC. It utilized conventional Bluetooth to implement remote control of the motorized bike’s speed during an exercise session as well as to provide real-time visualization of the exercise progress. We developed three customized tablet PC apps with similar user interfaces but different communication protocols for all the platforms to provide a graphical representation of exercise progress. The same microcontroller unit (MCU), ESP-32, was used in all the systems. Results Each system was tested in 1-minute exercise sessions at various speeds. To evaluate the accuracy of the measured data, in addition to reading speed values from the iBikE app, the cycling speed of the bikes was measured continuously using a tachometer. The mean differences of averaged RPMs for both data sets were calculated. The calculated values were 0.38 ± 0.03, 0.25 ± 0.27, and 6.7 ± 3.3 for the BLE system, the Wi-Fi system, and the conventional Bluetooth system, respectively. Conclusion All interfaces provided sufficient accuracy for use in telerehabilitation.
Opioid addiction is a serious public health problem in the US, and this study aimed to explore how natural language processing (NLP) can be used to identify factors that contribute to distress in individuals with opioid addiction, and then use this information along with structured data to predict the outcome of opioid treatment programs (OTP). The study analyzed medical records data and clinical notes of 1,364 patients, out of which 136 succeeded in the program and 1,228 failed. The results showed that several factors influenced the success of patients in the program, including sex, race, education, employment, secondary substance, tobacco use, and type of residences. XGBoost with down sampling was the best model. The accuracy of the model was 0.71 and the AUC score was 0.64. The study highlights the importance of using both structured and unstructured data to evaluate the effectiveness of OTP.