The purpose of this study was to evaluate the potential collective opportunities and challenges of transforming real-world data (RWD) to real-world evidence for clinical effectiveness by focusing on aligning analytic definitions of oncology end points. Patients treated with a qualifying therapy for advanced non-small cell lung cancer in the frontline setting meeting broad eligibility criteria were included to reflect the real-world population. Although a trend toward improved outcomes in patients receiving PD-(L)1 therapy over standard chemotherapy was observed in RWD analyses, the magnitude and consistency of treatment effect was more heterogeneous than previously observed in controlled clinical trials. The study design and analysis process highlighted the identification of pertinent methodological issues and potential innovative approaches that could inform the development of high-quality RWD studies.

naturemedicine and ethicists, is developing international, consensus-based guidelines for use by researchers and patient partners in preparing ethics submissions and for use by research ethics committees and institutional review boards in the assessment of PRO research.The guidelines will focus specifically on ethical considerations of PRO research and data collection in clinical practice, using methodological guideline development of the EQUATOR (Enhancing Quality and Transparency of Health Research) Network 10 .The development process will include a literature review, a modified Delphi exercise and an international consensus meeting involving members of research ethics committees, experts in research ethics, patient partners, trialists and PRO researchers.Given the dearth of guidance currently available, the authors plan to hold the Delphi exercise and consensus meeting with a view to publishing the guideline in 2021.

PURPOSE This study compared real-world end points extracted from the Cancer Analysis System (CAS), a national cancer registry with linkage to national mortality and other health care databases in England, with those from diverse US oncology data sources, including electronic health care records, insurance claims, unstructured medical charts, or a combination, that participated in the Friends of Cancer Research Real-World Evidence Pilot Project 1.0. Consistency between data sets and between real-world overall survival (rwOS) was assessed in patients with immunotherapy-treated advanced non–small-cell lung cancer (aNSCLC). PATIENTS AND METHODS Patients with aNSCLC, diagnosed between January 2013 and December 2017, who initiated treatment with approved programmed death ligand-1 (PD-[L]1) inhibitors until March 2018 were included. Real-world end points, including rwOS and real-world time to treatment discontinuation (rwTTD), were assessed using Kaplan-Meier analysis. A synthetic data set, Simulacrum, on the basis of conditional random sampling of the CAS data was used to develop and refine analysis scripts while protecting patient privacy. RESULTS Characteristics (age, sex, and histology) of the 2,035 patients with immunotherapy-treated aNSCLC included in the CAS study were broadly comparable with US data sets. In CAS, a higher proportion (46.7%) of patients received a PD-(L)1 inhibitor in the first line than in US data sets (18%-30%). Median rwOS (11.4 months; 95% CI, 10.4 to 12.7) and rwTTD (4.9 months; 95% CI, 4.7 to 5.1) were within the range of US-based data sets (rwOS, 8.6-13.5 months; rwTTD, 3.2-7.0 months). CONCLUSION The CAS findings were consistent with those from US-based oncology data sets. Such consistency is important for regulatory decision making. Differences observed between data sets may be explained by variation in health care settings, such as the timing of PD-(L)1 approval and reimbursement, and data capture.

In prior work, Friends of Cancer Research convened multiple data partners to establish standardized definitions for oncology real-world end points derived from electronic health records (EHRs) and claims data. Here, we assessed the performance of real-world overall survival (rwOS) from data sets sourced from EHRs by evaluating the ability of the end point to reflect expected differences from a previous randomized controlled trial across five data sources, after applying inclusion/exclusion criteria. The KEYNOTE-189 clinical trial protocol of platinum doublet chemotherapy (chemotherapy) vs. programmed cell death protein 1 (PD-1) in combination with platinum doublet chemotherapy (PD-1 combination) in first-line nonsquamous metastatic non-small cell lung cancer guided retrospective cohort selection. The Kaplan-Meier product limit estimator was used to calculate 12-month rwOS with 95% confidence intervals (CIs) in each data source. Cox proportional hazards models estimated hazard ratios (HRs) and associated 95% CIs, controlled for prognostic factors. Once the inclusion/exclusion criteria were applied, the five resulting data sets included 155 to 1,501 patients in the chemotherapy cohort and 36 to 405 patients in the PD-1 combination cohort. Twelve-month rwOS ranged from 45% to 58% in the chemotherapy cohort and 44% to 68% in the PD-1 combination cohort. The adjusted HR for death ranged from 0.80 (95% CI: 0.69, 0.93) to 1.15 (95% CI: 0.71, 1.85), controlling for age, gender, performance status, and smoking status. This study yielded insights regarding data capture, including ability of real-world data to precisely identify patient populations and the impact of criteria on end points. Sensitivity analyses could elucidate data set-specific factors that drive results.

PURPOSE: The proposed legislation Verifying Accurate and Leading-edge In vitro clinical test Development (VALID) clarifies the US Food and Drug Administration's authority to regulate laboratory-developed tests. Many stakeholders have pointed out that the lack of direct US Food and Drug Administration oversight has led to erroneous results that have serious patient consequences—in particular for patients with cancer. Technology Certification is a key provision proposed in VALID to navigate the balance between safety, patient access, and innovation; however, the maintenance cost of the proposed framework after implementation is unclear. METHODS: On the basis of 2019 retrospective data from a laboratory-developed test–based cancer diagnostics laboratory, we expressed laboratory complexity by the number and complexity of assays and in vitro diagnostic technologies. We estimated the national health care cost increase by modeling three stringencies of complying with the Act. We performed sensitivity analysis of our regulatory stringency model taking into account number of patients tested, materials, submission cost, and labor using extra cost per patient as the output. RESULTS: We estimate the national health care cost increase to range from $33M US dollars (USD) to $1,110M USD or $0.21 USD to $0.70 USD per employed person in the United States. Sensitivity analysis demonstrates that regulatory stringency is the primary driver of extra cost per patient. Cancer testing does not reflect all areas of in vitro diagnostics affected by VALID; nonetheless, concrete cost models are paramount in informing the ongoing legislative negotiations. CONCLUSION: Our findings show the critical importance of clarity in the legislative language to ensure balance between VALID's goals of assuring high-quality test performance and the burden to laboratories and overall health care cost.

e18749 Background: Friends of Cancer Research and five data partners used electronic health record-sourced datasets to conduct parallel analyses in real-world data (RWD). The study assessed estimates of rwOS among patients taking platinum doublet chemotherapy (chemo) or pembrolizumab in combination with chemo (IO+chemo) in first-line treatment for metastatic non-small cell lung cancer (mNSCLC). Methods: The KEYNOTE-189 clinical trial was selected as a reference for cohort selection in the RWD. We applied select available KEYNOTE-189 eligibility criteria to compare rwOS in a homogenous real-world population. Analyses were conducted following a mutually developed statistical analysis plan that was harmonized across data partners. Twelve-month rwOS, and unadjusted and adjusted hazard ratios (HR) for death were calculated using time-to-event methods with patients censored at last structured activity prior to the data cutoff. Results: The five datasets included patients treated with chemo (range of n=155 to 1501) and IO+chemo patients (n=36 to 405, Table). Twelve-month rwOS estimates ranged from 45% to 58% in the chemo arm and 44% to 68% in the IO +chemo arm. The HR for death comparing IO+chemo to chemo varied across groups, ranging from 0.79 (95% CI: 0.68, 0.92) to 1.10 (95% CI: 0.70, 1.72). The HR for death adjusted for age, gender, performance status, and smoking status did not significantly alter results, ranging from 0.80 (95% CI: 0.69, 0.93) to 1.15 (96% CI: 0.71, 1.85). Conclusions: The systematic evaluation of reproducibility and performance of rwOS across five real-world cohorts and the impact of implementing consistent eligibility criteria and methods was useful to understand the consistency of results in real-world populations and relationship between treatments and outcomes in the absence of randomization. A well-articulated research question and a pre-specified analytic plan that includes covariate definitions are necessary preliminary steps to guide data harmonization, standardization, and analysis. A range of observed results highlights the complexity of real-world data, including differences in sample size, underlying patient variability across datasets, and missing covariate data. The harmonization process and observed results underscore the importance of a well-articulated research question and a pre-specified analytic plan to guide data harmonization, standardization, and analysis. [Table: see text]

BACKGROUND:Tumor mutational burden (TMB) measurements aid in identifying patients who are likely to benefit from immunotherapy; however, there is empirical variability across panel assays and factors contributing to this variability have not been comprehensively investigated. Identifying sources of variability can help facilitate comparability across different panel assays, which may aid in broader adoption of panel assays and development of clinical applications.MATERIALS AND METHODS:Twenty-nine tumor samples and 10 human-derived cell lines were processed and distributed to 16 laboratories; each used their own bioinformatics pipelines to calculate TMB and compare to whole exome results. Additionally, theoretical positive percent agreement (PPA) and negative percent agreement (NPA) of TMB were estimated. The impact of filtering pathogenic and germline variants on TMB estimates was assessed. Calibration curves specific to each panel assay were developed to facilitate translation of panel TMB values to whole exome sequencing (WES) TMB values.RESULTS:Panel sizes >667 Kb are necessary to maintain adequate PPA and NPA for calling TMB high versus TMB low across the range of cut-offs used in practice. Failure to filter out pathogenic variants when estimating panel TMB resulted in overestimating TMB relative to WES for all assays. Filtering out potential germline variants at >0% population minor allele frequency resulted in the strongest correlation to WES TMB. Application of a calibration approach derived from The Cancer Genome Atlas data, tailored to each panel assay, reduced the spread of panel TMB values around the WES TMB as reflected in lower root mean squared error (RMSE) for 26/29 (90%) of the clinical samples.CONCLUSIONS:Estimation of TMB varies across different panels, with panel size, gene content, and bioinformatics pipelines contributing to empirical variability. Statistical calibration can achieve more consistent results across panels and allows for comparison of TMB values across various panel assays. To promote reproducibility and comparability across assays, a software tool was developed and made publicly available.

BACKGROUND:Diagnostic tests, including US Food and Drug Administration (FDA)-approved tests and laboratory-developed tests, are frequently used to guide care for patients with cancer, and, recently, have been the subject of several policy discussions and insurance coverage determinations. As the use of diagnostic testing has evolved, stakeholders have raised questions about the lack of standardized test performance metrics and the risk this poses to patients.OBJECTIVES:To describe the use of diagnostic testing for patients with advanced non-small-cell lung cancer (NSCLC), to analyze the utilization of FDA-approved versus laboratory-developed diagnostic tests, and to evaluate the impact of existing regulatory and coverage frameworks on diagnostic test ordering and physician treatment decision-making for patients with advanced NSCLC.METHODS:We conducted a 2-part study consisting of an online survey and patient chart review from March 1, 2019, to March 25, 2019, of physicians managing patients with advanced NSCLC. Respondents qualified for this study if they managed at least 5 patients with advanced NSCLC per month and had diagnosed at least 1 patient with advanced NSCLC in the 12 months before the survey. A total of 150 physicians completed the survey; before completing the survey, they were instructed to review between 4 and 8 charts of patients with stage IV NSCLC from their list of active patients.RESULTS:A total of 150 practicing oncologists who manage patients with advanced NSCLC responded to the survey and reviewed a total of 815 patient charts. Of these 815 patients, 812 (99.6%) were tested for at least 1 biomarker, including 73% of patients who were tested for EGFR, 70% tested for ALK, 58% tested for BRAF V600E, and 38% of patients tested for ROS1, by FDA-approved diagnostic tests. In all, 185 (83%) patients who tested positive for EGFR and 60 (83%) patients who tested positive for ALK received an FDA-approved targeted therapy for their biomarker. A total of 98 (65%) physicians responded that the patient's insurance coverage factored into their decision to order diagnostic tests and 69 (45%) physicians responded that cost or the patient's insurance coverage could influence them not to prescribe an indicated targeted therapy.CONCLUSION:The survey results indicate that diagnostic testing has become routine in the treatment of patients with advanced NSCLC, the use of FDA-approved diagnostic tests has increased, and insurance coverage and cost influence patient access to diagnostic testing as well as to targeted treatment options.

The field of cell therapy is rapidly emerging as a priority area for oncology research and drug development. Currently, two chimeric antigen receptor T-cell therapies are approved by the US Food and Drug Administration and other agencies worldwide for two types of hematologic cancers. To facilitate the development of these therapies for patients with life-threatening cancers with limited or no therapeutic options, science- and risk-based approaches will be critical to mitigating and balancing any potential risk associated with either early clinical research or more flexible manufacturing paradigms. Friends of Cancer Research and the Parker Institute for Cancer Immunotherapy convened an expert group of stakeholders to develop specific strategies and proposals for regulatory opportunities to accelerate the development of cell therapies as promising new therapeutics. This meeting took place in Washington, DC on May 17, 2019. As academia and industry expand research efforts and cellular product development pipelines, this report summarizes opportunities to accelerate entry into the clinic for exploratory studies and optimization of cell products through manufacturing improvements for these promising new therapies.

e19270 Background: Friends of Cancer Research convened 9 data partners to identify data elements and common definitions for real world (rw) endpoints to evaluate populations typically excluded from clinical trials. Here we report on rwOS by frontline treatment and comorbidities. Methods: A retrospective observational analysis of patients with aNSCLC initiating frontline platinum doublet chemotherapy (chemo) or PD-(L)1-based immuno-oncologic (IO) therapy (monotherapy or chemo combination) between 1 Jan 2011 to 31 Mar 2018 was conducted using administrative claims, EHR, and cancer registry RWD. We evaluated rwOS from frontline therapy initiation using Kaplan-Meier methods, stratified by ECOG status, brain metastases (ICD), history of chronic kidney or liver disease (CKD/ CLD, ICD), and evidence of kidney or liver dysfunction (KD/ LD, lab-based). Results: A total of 33,649 patients were included (N 972-17,454) with 10 to 26% of patients receiving IO as frontline therapy. There was a broad range of comorbidity prevalence across datasets and patients with evidence of comorbidity had comparatively shorter 12-month OS (Table). Conclusions: RWD analyses can generate expanded evidence on patient outcomes for populations routinely excluded from clinical trials and may help inform decision making where sparse data exist on appropriate treatment approaches. Additional understanding of data missingness, sensitivity of definitions, and covariate adjustment are needed to make direct comparisons across regimens and data sources. [Table: see text]

e14124 Background: VALID Act is a bipartisan draft legislation proposing to ensure the quality and safety of diagnostic tests through direct FDA oversight. Currently, FDA exercises enforcement discretion and exempts from premarket review tests that are developed and used in the same clinical laboratory. Many stakeholders have pointed out that the lack of direct FDA oversight has led to erroneous results that have serious patient consequences. However, the maintenance cost after the implementation of the Act is unknown. Thus, we estimate the additional financial burden of genotyping all new late stage cancer patients under a fully implemented VALID Act precertification framework. Methods: Based on our laboratory with 36 high-complexity clinical assays, we modeled the cost increase by dividing the total anticipated cost for annual FDA precertification by our 2019 patient volume. To model the stringency of complying with the Act, we calculated either 3, 6, or 9 assays as representative of the laboratory’s test complexity. To make the estimated additional annual national healthcare cost relatable, we expressed the increase as a distributed cost per newly diagnosed late stage cancer patient in 2019 (NCI SEER) or per employed person in 2019 (Bureau of Labor Statistics). Results: FDA precertification for 3 assays was estimated at $638,310.13, or when divided by our annual late stage patient volume (n = 10,232), an increased cost of $62.38/patient. Precertification for 6 or 9 assays costs $684,751.60 (+$66.92/patient) or $1,253,717.25 (+$122.53/patient), respectively. If only 1 cancer center per state/DC/PR got precertified (n = 52), multiplied by the cost of precertifying 3 assays, then the annual national healthcare cost would increase by $33,192,126.50, or +$42.37/new late stage patient (n = 783,476) or +$0.21/employed (n = 158,803,000). Precertification for 52 centers for 6 or 9 assays would cost $35,607,083.20 (+$45.45/patient, +$0.22/employed) or $65,193,297.00 (+$83.21/patient, +$0.41/employed), respectively. Nationally, there are ~886 centers and the increased cost was estimated at < $7/employed. Even though the VALID Act will affect more than late stage cancer patients, thus costs will vary, our data show how a specific change of the Act will impact oncology practice. Conclusions: We provided a concrete cost model for late stage cancer patients under the proposed VALID Act. Cost modeling is important in informing the balanced legislative language needed to ensure patient access to validated cancer diagnostics, while supporting continued innovation by clinical laboratories.

e19311 Background: Leveraging data from a collaboration with 9 data partners, Friends of Cancer Research convened the Real-world Evidence Pilot 2.0, to examine trends and real world (rw) data endpoints in immunotherapy (IO) use for the front line treatment of aNSCLC. Methods: This study leveraged parallel analyses of rw data elements across heterogenous data sources (EHR, administrative claims, and registry) to: a) describe trends in uptake and use of novel IO frontline therapy after advanced diagnosis in NSCLC patients treated in usual care settings and b) examine associations between treatment and rw outcomes at one-year follow-up. The proportion of patients treated on each regimen (IO single agent, chemo, or IO + chemo) from 2011 through 2017 were calculated. Analysis included proportion of patients across treatment regimens stratified by year to describe post approval uptake of IO. Kaplan-Meier survival estimates were reported to adjust for follow-up time and stratified by PD-L1 status and stage. Results: Seven datasets identified a range of 999 to 4617 patients per dataset for this analysis. Across datasets, 2508, 3446, and 4176 patients initiated treatment in 2015, 2016, and 2017, respectively. No patients received IO or IO + chemo regimens prior to 2015. Initial approvals for IO use in aNSCLC occurred in October 2015 and for first line in metastatic NSCLC in October 2016. When examining survival at 1 year, overall, OS in PD-(L)1 + patients appeared longer than those with a PD-(L)1 - status. Conclusions: RWE analyses may reveal important trends in clinical cancer patient care including patterns of off-label use. The heterogeneity in the timing of IO uptake across datasets ranged from immediately after approval to ~12 months post-approval. [Table: see text]

Patient reported outcomes (PROs) are the gold standard for assessing patients’ experience of treatment in oncology, defined in the 21st Century Cures Act as information about patients’ experiences with a disease or condition, including the impact of a disease or condition, or a related therapy or clinical investigation on patients’ lives; and patient preferences with respect to treatment of their disease or condition [ 1 ]. PROs provide a comprehensive assessment of the benefits and risks of new medical products, as well as essential data to inform real-world use. Although RCTs are the ultimate source for information for evaluating products in development, they are not always feasible for rare diseases with few or no effective treatment options available. Thus, it is important to consider other measures that can help to improve the strength of evidence for cell and gene therapies targeting rare indications. While collection of PROs and other patient experience endpoints does not resolve the difficulty of conducting trials in small populations, doing so contributes empirical evidence that informs both product development and patient access. Additionally, including routine collection of PROs in registries may provide supplemental data to further characterize the benefit:risk profile of cell and gene therapies at follow-up times that would be infeasible to operationalize in a clinical trial setting.

Unlocking the full potential of pathology data by gaining computational access to histological pixel data and metadata (digital pathology) is one of the key promises of computational pathology. Despite scientific progress and several regulatory approvals for primary diagnosis using whole-slide imaging, true clinical adoption at scale is slower than anticipated. In the U.S., advances in digital pathology are often siloed pursuits by individual stakeholders, and to our knowledge, there has not been a systematic approach to advance the field through a regulatory science initiative. The Alliance for Digital Pathology (the Alliance) is a recently established, volunteer, collaborative, regulatory science initiative to standardize digital pathology processes to speed up innovation to patients. The purpose is: (1) to account for the patient perspective by including patient advocacy; (2) to investigate and develop methods and tools for the evaluation of effectiveness, safety, and quality to specify risks and benefits in the precompetitive phase; (3) to help strategize the sequence of clinically meaningful deliverables; (4) to encourage and streamline the development of ground-truth data sets for machine learning model development and validation; and (5) to clarify regulatory pathways by investigating relevant regulatory science questions. The Alliance accepts participation from all stakeholders, and we solicit clinically relevant proposals that will benefit the field at large. The initiative will dissolve once a clinical, interoperable, modularized, integrated solution (from tissue acquisition to diagnostic algorithm) has been implemented. In times of rapidly evolving discoveries, scientific input from subject-matter experts is one essential element to inform regulatory guidance and decision-making. The Alliance aims to establish and promote synergistic regulatory science efforts that will leverage diverse inputs to move digital pathology forward and ultimately improve patient care.