Objectives:Medical product safety surveillance efforts, whether using electronic health record (EHR) or claims data, typically rely on structured codes. Utilizing unstructured EHR data, particularly information extracted from clinical text through natural language processing (NLP), enriches information available for data mining, phenotyping, and surveillance. To assess overlapping and distinct information across structured and unstructured EHR data, we mapped both to a common vocabulary (Medical Dictionary for Regulatory Activities, MedDRA). We assess the feasibility of implementing such a mapping and explored similarities and differences at multiple levels of the concept hierarchy. Materials and Methods:We randomly sampled 15,000 encounters (5000 each from ambulatory, emergency, and inpatient encounters). For each encounter, we extracted MedDRA concepts from clinical notes using MetaMap and mapped structured ICD-10-CM diagnoses to MedDRA. We evaluated corroboration between data sources across the MedDRA hierarchy, as well as the unique information contributed by each source. Results:We processed 119,492 clinical notes and mapped 163,254 ICD-10-CM codes to MedDRA. Most encounters (73-98%) had some overlap between MedDRA preferred terms identified from structured and unstructured data. Among MedDRA concepts found in unstructured text, 80-95% were not found in the encounter's associated ICD-10-CM coded data. Discussion and Conclusion:While MedDRA concepts from structured data were mostly corroborated by those extracted from unstructured clinical text, the majority of MedDRA concepts recognized in each encounter were only mentioned in text. Leveraging MedDRA-encoded unstructured text can provide a more comprehensive clinical picture of patients and complement the structured data traditionally used in epidemiological and pharmacovigilance studies.

OBJECTIVE:To assess differences in lung cancer screening care satisfaction among Veterans residing in rural and nonrural areas. METHODS:We identified Veterans screened for lung cancer from 2018-2021 at 10 Veterans Affairs Medical Centers and surveyed a sample using an adapted version of the Patient Satisfaction with Cancer Care (PSCC) scale. The primary analysis ( t test) evaluated the association between rurality and care satisfaction in responder and population-weighted cohorts. Secondary analysis (linear regression) estimated the association of age, sex, race, ethnicity, smoking status, education attainment, and facility location with care satisfaction. The Fisher exact test evaluated the association of rurality and smoking status with individual PSCC item agreement. RESULTS:The response rate was 36.0% (n=689 complete surveys). The mean responder age was 68.3 years; 95.4% were male; 50.2% rural; 55.6% currently smoking. There was no significant difference in care satisfaction by rurality in the responder cohort [mean PSCC score 72.4 ( SD =14.2), rural 72.4, non-rural 72.4, P =0.99] or population-weighted cohort [mean PSCC score 71.9 ( SD =14.2), rural 72.1, non-rural 71.9, P =0.81]. "Currently smoking" status was negatively associated with care satisfaction [coefficient=-3.20 (-6.42,0.02)]. Individual PSCC items with the highest and lowest agreement were "I felt that I was treated with courtesy and respect" (93.6% agree); "I knew what the next step in my care would be" (65.2% agree). CONCLUSIONS:Veterans in rural and nonrural areas reported similar levels of satisfaction with lung cancer screening care. Veterans who currently smoke reported lower levels of satisfaction. Veterans identified limited understanding of lung cancer screening processes and poor communication as areas needing improvement.

Objectives:Scalable computable phenotyping algorithms are critical for conducting high-throughput disease-outcome research in large, distributed-data electronic health record (EHR) and claims data settings. We developed and evaluated a claims- and EHR-based computable phenotyping algorithm for anaphylaxis, a rare acute condition that is challenging to accurately identify using claims data alone. Materials and Methods:Potential anaphylaxis events came from two healthcare systems (Kaiser Permanente Washington [KPWA] and Vanderbilt University Medical Center [VUMC]). We engineered features from clinical text using automated natural language processing (NLP) methods. We then developed a phenotyping algorithm using four NLP- and diagnosis code-based silver labels (proxies for the gold-standard labels). Gold-standard abstracted outcomes were used to evaluate algorithm performance. Results:The largest area under the receiver operating characteristic curve (AUC) was 0.931 for an NLP-based silver-label model at KPWA. Depending on the model and healthcare system site, positive predictive value (PPV) and sensitivity at the threshold of predicted probability that maximized F1 score ranged from 0.52 to 0.77 (PPV) and 0.78 to 1 (sensitivity). Discussion:NLP-based silver-label models had large AUC at KPWA but not at VUMC. This may be because clinical text at KPWA is only available for outpatient encounters and secure messaging. High sensitivity for identifying anaphylaxis can be obtained using our best-performing models. Conclusion:The best-performing models had better PPV and sensitivity tradeoffs than prior bespoke anaphylaxis models with costly, manually curated features. The simplicity of the approach compared to traditional phenotyping methods allows it to be deployed easily at multiple health care systems.

PURPOSE:Tobacco use is not commonly represented as computable information in the electronic health record (EHR). We developed an algorithm in the Veterans Health Administration (VHA) to identify tobacco ever-use among Veterans. METHODS:We used the VHA corporate data warehouse to develop an algorithm comprised of multiple data types (health factors [semi-structured template data entry and decision support tools], billing, orders, medication, and encounter codes) to identify tobacco ever-use (current or former) versus never use. Algorithm accuracy was compared to two reference standards: (1) EHR abstraction cohort and (2) Veteran self-reported survey cohort. We calculated the sensitivity and positive predictive values (PPV) for the algorithm and stratified by its data types for the EHR abstraction cohort. We calculated the sensitivity, specificity, PPV, and negative predictive value (NPV) for the algorithm and stratified by its data types for the survey cohort. RESULTS:The algorithm correctly identified 424 of 426 individuals with tobacco ever-use when compared to data abstracted from the EHR: sensitivity 1.00 (95 % CI 0.98-1.00); PPV 1.00 (95 % CI 0.98-1.00). Compared to survey data, the algorithm correctly identified 514 of 547 participants with tobacco ever-use: sensitivity 0.94 (95 % CI 0.92-0.96); PPV 0.88 (95 % CI 0.85-0.91). The specificity was 0.53 (95 % CI 0.45-0.62), and NPV of 0.70 (95 % CI 0.61-0.79). Of all data types, health factors had the highest sensitivity in both cohorts. CONCLUSIONS:This novel tool had excellent sensitivity and PPV for tobacco ever-use in two cohorts. Future research should study this tool to support preventive healthcare services.

Objectives To evaluate the feasibility for use of electronic health record (EHR) data in conducting adverse event surveillance among women who received mid-urethral slings (MUS) to treat stress urinary incontinence (SUI) in five health systems.Design Retrospective observational study using EHR data from 2010 through 2021. Women with a history of MUS were identified using common data models; a common analytic code was executed at each site. A manual chart review was conducted in a per-site random patient subset to establish a reference standard. Automated text processing (Text Processed Integrated (TPI)) was developed and evaluated at each site to determine the surgical approach and synthetic mesh implantation. Patients were characterized and surgical outcomes were ascertained over 730 subsequent days.Setting Five large tertiary care academic medical centers.Participants Across five health systems, 9,906 eligible patients (mean age 57–60 per site) were identified.Main outcome measures Determination of surgical approach, synthetic mesh implantation, and assessment of the duration of surveillance for mortality and reoperation rates following MUS implantation.Results In the TPI cohort analysis, 3,331 patients were identified. Surgical approach per site was retropubic (42% to 77%), transobturator (6% to 44%), single incision (0% to 24%), and adjustable sling (0% to <4%). Concordance rates for TPI using chart review were 71%–90% at each site for the surgical approach and 28%–85% for synthetic mesh implantation. Patient follow-up observation rates for mortality and reoperation ranged from 22% to 36% at 90 days, 15% to 30% at 365 days, and 8% to 19% at 730 days.Conclusion Using EHR data alone, identification of medical devices and surgical approaches was feasible among women with MUS surgery for SUI, but long-term follow-up ascertainment rates were low. Medical device surveillance using EHR data should be evaluated in the context of the clinical use case, as applicability may vary.

Introduction: Lung cancer screening is underutilized, especially in rural areas where lung cancer mortality is high. Approximately 11.2% of the U.S. population over age 50 years meet the U.S. Preventive Services Task Force (USPSTF) 2021 lung cancer screening eligibility criteria; the proportion of eligible Veterans is unknown. This study evaluated the proportion of Veterans who are USPSTFeligible and tested the hypothesis that more USPSTF 2021-eligible Veterans reside in rural versus nonrural areas. Methods: Investigators cross-sectionally surveyed a national sample of Veterans aged 50 years and older January-November 2022. Oversampling ensured inclusion of minority groups and accounted for geographic variation in tobacco use. Analyses in 2023-2024 evaluated the proportion of USPSTF-eligible Veterans by year (2013 and 2021) and tested USPSTF-2021 eligibility by rural status (rural versus nonrural) using chi square tests. Weighting accounted for survey nonresponse and applied results to the whole Veteran population in a sensitivity analysis. Results: Of 2,000 surveyed, 754 responded (37.7% response rate); most respondents were White (74.4%), male (92.6%), and resided in nonrural areas (66.0%). Proportions meeting USPSTF criteria were 35.5% (95% CI=31.6%, 39.6%) in 2021 and 27.8% (95% CI=24.3%, 31.7%) in 2013. The proportion of USPSTF 2021-eligible rural Veterans (41.2%; 95% CI=34.8%, 48.0%) was higher compared with nonrural (32.5%; 95% CI=27.7%, 37.7%), p=0.037. A sensitivity analysis found the proportion of Veterans USPSTF 2021 eligible in the whole population was 33.0%. Conclusions: The proportion of Veterans USPSTF2021 eligible was nearly 3 times higher than the general U.S. population (11.2%), and a greater proportion of eligible Veterans resided in rural compared with nonrural areas. These findings are critical for policies aimed at fully implementing lung cancer screening at scale.

OBJECTIVES:Automated phenotyping algorithms can reduce development time and operator dependence compared to manually developed algorithms. One such approach, PheNorm, has performed well for identifying chronic health conditions, but its performance for acute conditions is largely unknown. Herein, we implement and evaluate PheNorm applied to symptomatic COVID-19 disease to investigate its potential feasibility for rapid phenotyping of acute health conditions. MATERIALS AND METHODS:PheNorm is a general-purpose automated approach to creating computable phenotype algorithms based on natural language processing, machine learning, and (low cost) silver-standard training labels. We applied PheNorm to cohorts of potential COVID-19 patients from 2 institutions and used gold-standard manual chart review data to investigate the impact on performance of alternative feature engineering options and implementing externally trained models without local retraining. RESULTS:Models at each institution achieved AUC, sensitivity, and positive predictive value of 0.853, 0.879, 0.851 and 0.804, 0.976, and 0.885, respectively, at quantiles of model-predicted risk that maximize F1. We report performance metrics for all combinations of silver labels, feature engineering options, and models trained internally versus externally. DISCUSSION:Phenotyping algorithms developed using PheNorm performed well at both institutions. Performance varied with different silver-standard labels and feature engineering options. Models developed locally at one site also worked well when implemented externally at the other site. CONCLUSION:PheNorm models successfully identified an acute health condition, symptomatic COVID-19. The simplicity of the PheNorm approach allows it to be applied at multiple study sites with substantially reduced overhead compared to traditional approaches.

PURPOSE:The US Food and Drug Administration's Sentinel Innovation Center aimed to establish a query-ready, quality-checked distributed data network containing electronic health records (EHRs) linked with insurance claims data for at least 10 million individuals to expand the utility of real-world data for regulatory decision-making. METHODS:In this report, we describe the resulting network, the Real-World Evidence Data Enterprise (RWE-DE), including data from two commercial EHR-claims linked assets collectively termed the Commercial Network covering 21 million lives, and four academic partner institutions collectively termed the Development Network covering 4.5 million lives. RESULTS:We discuss provenance and completeness of the data converted in the Sentinel Common Data Model (SCDM), describe patient populations, and report on EHR-claims linkage characterization for all contributing data sources. Further, we introduce a standardized process to store free-text notes in the Development Network for efficient retrieval as needed. CONCLUSIONS:Finally, we outline typical use cases for the RWE-DE where it can broaden the reach of the types of questions that can be addressed by the Sentinel system.

INTRODUCTION:Lung cancer screening is widely underutilized. Organizational factors, such as readiness for change and belief in the value of change (change valence), may contribute to underutilization. The aim of this study was to evaluate the association between healthcare organizations' preparedness and lung cancer screening utilization. METHODS:Investigators cross-sectionally surveyed clinicians, staff, and leaders at10 Veterans Affairs from November 2018 to February 2021 to assess organizational readiness to implement change. In 2022, investigators used simple and multivariable linear regression to evaluate the associations between facility-level organizational readiness to implement change and change valence with lung cancer screening utilization. Organizational readiness to implement change and change valence were calculated from individual surveys. The primary outcome was the proportion of eligible Veterans screened using low-dose computed tomography. Secondary analyses assessed scores by healthcare role. RESULTS:The overall response rate was 27.4% (n=1,049), with 956 complete surveys analyzed: median age of 49 years, 70.3% female, 67.6% White, 34.6% clinicians, 61.1% staff, and 4.3% leaders. For each 1-point increase in median organizational readiness to implement change and change valence, there was an associated 8.4-percentage point (95% CI=0.2, 16.6) and a 6.3-percentage point increase in utilization (95% CI= -3.9, 16.5), respectively. Higher clinician and staff median scores were associated with increased utilization, whereas leader scores were associated with decreased utilization after adjusting for other roles. CONCLUSIONS:Healthcare organizations with higher readiness and change valence utilized more lung cancer screening. These results are hypothesis generating. Future interventions to increase organizations' preparedness, especially among clinicians and staff, may increase lung cancer screening utilization.