Background: The availability of high fidelity electronic health record (EHR) data is a hallmark of the learning health care system. Washington State’s Surgical Care Outcomes and Assessment Program (SCOAP) is a network of hospitals participating in quality improvement (QI) registries wherein data are manually abstracted from EHRs. To create the Comparative Effectiveness Research and Translation Network (CERTAIN), we semi-automated SCOAP data abstraction using a centralized federated data model, created a central data repository (CDR), and assessed whether these data could be used as real world evidence for QI and research. Objectives: Describe the validation processes and complexities involved and lessons learned. Methods: Investigators installed a commercial CDR to retrieve and store data from disparate EHRs. Manual and automated abstraction systems were conducted in parallel (10/2012-7/2013) and validated in three phases using the EHR as the gold standard: 1) ingestion, 2) standardization, and 3) concordance of automated versus manually abstracted cases. Information retrieval statistics were calculated. Results: Four unaffiliated health systems provided data. Between 6 and 15 percent of data elements were abstracted: 51 to 86 percent from structured data; the remainder using natural language processing (NLP). In phase 1, data ingestion from 12 out of 20 feeds reached 95 percent accuracy. In phase 2, 55 percent of structured data elements performed with 96 to 100 percent accuracy; NLP with 89 to 91 percent accuracy. In phase 3, concordance ranged from 69 to 89 percent. Information retrieval statistics were consistently above 90 percent. Conclusions: Semi-automated data abstraction may be useful, although raw data collected as a byproduct of health care delivery is not immediately available for use as real world evidence. New approaches to gathering and analyzing extant data are required.

Introduction:New medical technologies are an important part of delivering innovative healthcare, however, expanding use of medical technology is a major contributor to rising costs. The increase in medical spending is related to new technologies being rapidly developed, marketed and adopted; and often being incorporated into health systems with little evidence. They also come with higher prices when compared to existing technologies.Methods:We describe how University of Washington (UW) Medicine has designed, and developed a new hospital-based health technology assessment (HB-HTA) program, Smart Innovation. Smart Innovation will replace a fragmented and complex set of purchasing and coverage-decision processes. The program will streamline the decision-making process for new medical technologies and balance quick turnaround with rigorous evidence standards. The program is also being developed in collaboration with UW Medicine's Value Analysis team, an evidence-based purchasing team and MedApproved, a new centralized software program for medical purchasing at UW Medicine.Results:Smart Innovation has been reviewing technologies during its first year and has received encouraging results. For example, by adopting a new liver ablation technology, UW Medicine has estimated improved patient outcomes by reducing the number of procedures and adverse events; as well as saving approximately USD 8,000 per patient. Additionally, The Smart Innovation program has achieved projected cost avoidance from deciding not to adopt uncertain or investigational technologies. For example, by not adopting a new bladder cancer screen, our models indicate UW Medicine will avoid spending USD 1.5 million per year.Conclusions:Smart Innovation is proving to be an effective program for reviewing and making critical healthcare policy decisions that is showing significant fiscal and patient improvements for UW Medicine. As the program continues to grow and become embedded into UW Medicine, its impacts will become even more valuable and system-wide.

Importance Intermittent claudication (IC) is the most common presentation of infrainguinal peripheral artery disease. Both medical and revascularization interventions for IC aim to increase walking comfort and distance, but there is inconclusive evidence of the comparative benefit of revascularization given the possible risk of limb loss. Objective To compare the effectiveness of a medical (walking program, smoking cessation counseling, and medications) vs revascularization (endovascular or surgical) intervention for IC in the community, focusing on outcomes of greatest importance to patients. Design, Setting, and Participants Longitudinal (12-month follow-up) prospective observational cohort study conducted between July 3, 2011, and November 5, 2014, at 15 clinics associated with 11 hospitals in Washington State. Participants were 21 years or older with newly diagnosed or established IC. Interventions Medical or revascularization interventions. Main Outcomes and Measures Primary end points were 12-month change scores on the distance, speed, and stair-climb domains of the Walking Impairment Questionnaire (score range, 0-100). Secondary outcomes were change scores on the Walking Impairment Questionnaire pain domain (score range, 0-100), Vascular Quality of Life Questionnaire (VascuQol) (score range, 1-7), European Quality of Life-5 Dimension Questionnaire (EQ-5D) (score range, 0-1), and Claudication Symptom Instrument (CSI) (score range, 0-4). Results A total of 323 adults were enrolled, with 282 (87.3%) in the medical cohort. At baseline, the mean duration of disease was longer for participants in the medical cohort, while those in the revascularization cohort reported more severe disease. Other characteristics were well balanced. At 12 months, change scores in the medical cohort reached significance for the following 3 outcomes: speed (5.9; 95% CI, 0.5-11.3; P = .03), VascuQol (0.28; 95% CI, 0.08-0.49; P = .008), and EQ-5D (0.038; 95% CI, 0.011-0.066; P = .006). In the revascularization cohort, there were significant improvements in the following 7 outcomes: distance (19.5; 95% CI, 7.9-31.0; P = .001), speed (12.1; 95% CI, 1.4-22.8; P = .03), stair climb (11.4; 95% CI, 1.3-21.5; P = .03), pain (20.7; 95% CI, 11.0-30.4; P < .001), VascuQol (1.10; 95% CI, 0.80-1.41; P < .001), EQ-5D (0.113; 95% CI, 0.067-0.159; P < .001), and CSI (-0.63; 95% CI, -0.96 to -0.31; P < .001). Relative improvements (percentage changes) at 12 months in the revascularization cohort over the medical cohort were observed as follows: distance (39.1%), speed (15.6%), stair climb (9.7%), pain (116.9%), VascuQol (41%), EQ-5D (18%), and CSI (13.5%). Conclusions and Relevance Among patients with IC, those in the revascularization cohort had significantly improved function (Walking Impairment Questionnaire), better health-related quality of life (VascuQol and EQ-5D), and fewer symptoms (CSI) at 12 months compared with those in the medical cohort, providing important information to inform treatment strategies in the community.

INTRODUCTION:Delivering more appropriate, safer, and highly effective health care is the goal of a learning health care system. The Agency for Healthcare Research and Quality (AHRQ) funded enhanced registry projects: (1) to create and analyze valid data for comparative effectiveness research (CER); and (2) to enhance the ability to monitor and advance clinical quality improvement (QI). This case report describes barriers and solutions from one state-wide enhanced registry project.METHODS:The Comparative Effectiveness Research and Translation Network (CERTAIN) deployed the commercially available Amalga Unified Intelligence System™ (Amalga) as a central data repository to enhance an existing QI registry (the Automation Project). An eight-step implementation process included hospital recruitment, technical electronic health record (EHR) review, hospital-specific interface planning, data ingestion, and validation. Data ownership and security protocols were established, along with formal methods to separate data management for QI purposes and research purposes. Sustainability would come from lowered chart review costs and the hospital's desire to invest in the infrastructure after trying it.FINDINGS:CERTAIN approached 19 hospitals in Washington State operating within 12 unaffiliated health care systems for the Automation Project. Five of the 19 completed all implementation steps. Four hospitals did not participate due to lack of perceived institutional value. Ten hospitals did not participate because their information technology (IT) departments were oversubscribed (e.g., too busy with Meaningful Use upgrades). One organization representing 22 additional hospitals expressed interest, but was unable to overcome data governance barriers in time. Questions about data use for QI versus research were resolved in a widely adopted project framework. Hospitals restricted data delivery to a subset of patients, introducing substantial technical challenges. Overcoming challenges of idiosyncratic EHR implementations required each hospital to devote more IT resources than were predicted. Cost savings did not meet projections because of the increased IT resource requirements and a different source of lowered chart review costs.DISCUSSION:CERTAIN succeeded in recruiting unaffiliated hospitals into the Automation Project to create an enhanced registry to achieve AHRQ goals. This case report describes several distinct barriers to central data aggregation for QI and CER across unaffiliated hospitals: (1) competition for limited on-site IT expertise, (2) concerns about data use for QI versus research, (3) restrictions on data automation to a defined subset of patients, and (4) unpredictable resource needs because of idiosyncrasies among unaffiliated hospitals in how EHR data are coded, stored, and made available for transmission-even between hospitals using the same vendor's EHR. Therefore, even a fully optimized automation infrastructure would still not achieve complete automation. The Automation Project was unable to align sufficiently with internal hospital objectives, so it could not show a compelling case for sustainability.

BACKGROUND:Learning health care systems apply the experiences of prior patients to inform care and help to guide decision making for current patients. These systems should help to deliver more effective, efficient, and appropriate care. Most examples of learning systems derive from integrated care delivery systems and examples of such systems in the community at large have been lacking.METHODS:The comparative effectiveness research translation network (CERTAIN) is a learning system bringing together hospitals and outpatient clinics across Washington State. CERTAIN leverages existing medical record-based data collection taking place at nearly all statewide hospitals and links this data collection with patient-reported information about function and quality of life.RESULTS:We have described the components of the CERTAIN infrastructure, the elements of a pilot project evaluating treatments of claudication, and the opportunities and challenges of developing and implementing a "real world" learning system. Examples in the areas of vascular disease, spine care, gastrointestinal disease, and urology.CONCLUSION:Learning health care systems face many operational challenges but hold great promise for discovery and implementation of more effective clinical practices.

Background: The field of clinical research informatics includes creation of clinical data repositories (CDRs) used to conduct quality improvement (QI) activities and comparative effectiveness research (CER). Ideally, CDR data are accurately and directly abstracted from disparate electronic health records (EHRs), across diverse health-systems.Objective: Investigators from Washington State’s Surgical Care Outcomes and Assessment Program (SCOAP) Comparative Effectiveness Research Translation Network (CERTAIN) are creating such a CDR. This manuscript describes the automation and validation methods used to create this digital infrastructure.Methods: SCOAP is a QI benchmarking initiative. Data are manually abstracted from EHRs and entered into a data management system. CERTAIN investigators are now deploying Caradigm’s Amalga™ tool to facilitate automated abstraction of data from multiple, disparate EHRs. Concordance is calculated to compare data automatically to manually abstracted. Performance measures are calculated between Amalga and each parent EHR. Validation takes place in repeated loops, with improvements made over time. When automated abstraction reaches the current benchmark for abstraction accuracy - 95% - itwill ‘go-live’ at each site.Progress to Date: A technical analysis was completed at 14 sites. Five sites are contributing; the remaining sites prioritized meeting Meaningful Use criteria. Participating sites are contributing 15-18 unique data feeds, totaling 13 surgical registry use cases. Common feeds are registration, laboratory, transcription/dictation, radiology, and medications. Approximately 50% of 1,320 designated data elements are being automatically abstracted – 25% from structured data; 25% from text mining.Conclusion: In semi-automating data abstraction and conducting a rigorous validation, CERTAIN investigators will semi-automate data collection to conduct QI and CER, while advancing the Learning Healthcare System.

Objective: To describe the inaugural comparative effectiveness research (CER) cohort study of Washington State's Comparative Effectiveness Research Translation Network (CERTAIN), which compares invasive with noninvasive treatments for peripheral artery disease, and to focus on the patient centeredness of this cohort study by describing it within the context of a newly published conceptual framework for patient-centered outcomes research (PCOR).Study Design and Setting: The peripheral artery disease study was selected because of clinician-identified uncertainty in treatment selection and differences in desired outcomes between patients and clinicians. Patient centeredness is achieved through the "Patient Voices Project," a CERTAIN initiative through which patient-reported outcome (PRO) instruments are administered for research and clinical purposes, and a study-specific patient advisory group where patients are meaningfully engaged throughout the life cycle of the study. A clinician-led research advisory panel follows in parallel.Results: Primary outcomes are PRO instruments that measure function, health-related quality of life, and symptoms, the latter developed with input from the patients. Input from the patient advisory group led to revised retention procedures, which now focus on short-term (3-6 months) follow-up. The research advisory panel is piloting a point-of-care, patient assessment checklist, thereby returning study results to practice. The cohort study is aligned with the tenets of one of the new conceptual frameworks for conducting PCOR.Conclusion: The CERTAIN's inaugural cohort study may serve as a useful model for conducting PCOR and creating a learning health care network. (C) 2013 Elsevier Inc. All rights reserved.

BACKGROUND: Little is known about surgeon characteristics associated with common bile duct injury (CBDI) during laparoscopic cholecystectomy (LC). Risk-taking preferences call influence physician behavior and practice. We evaluated self-reported differences in characteristics and risk-taking preference among surgeons with and without a reported history of CBDI.STUDY DESIGN: A mailed survey was sent to 4,100 general surgeons randomly selected from the mailing list of the American College of Surgeons. Surveys with a valid exclusion (retired, 110 LC experience) were considered responsive, but were excluded from data analysis.RESULTS: Forty-four percent responded (1,412 surveys analyzed), 37.7% reported being the primary surgeon when a CBDI occurred, and 12.9% had more than one injury. Surgeons reporting an injury were slightly older (52.8 +/- 9.0 years versus 51.3 +/- 9.8 years; p < 0.004) and in practice longer (20.8 +/- 9.7 years versus 18.9 +/- 10.5 years; p < 0.001). Surgeons not reporting a CBDI were more likely trained in LC during residency (63.3% versus 55.4% injuring) as compared with surgeons reporting a CBDI, who were more likely trained at an LC course (29.8% versus 38.2%). Surgeons in academic practice or who work with residents had lower reported rates of CBDI (7.9% versus 14.5% [academics]; 18.7% versus 25.0% [residents]). Mean risk score was 12.4 +/- 4.4 (range 6 to 30 [30 = highest]) with a similar average between those who did (12.2 +/- 4.5) and did not (11.9 +/- 4.4) report a CBDI (p < 0.23). Compared with surgeons in the lowest three deciles of risk score, relative risk for CBDI among surgeons in the upper three deciles was 17% greater (p = 0.07).CONCLUSIONS: More years performing LC and certain practice characteristics were associated with an increased rate of CBDI. The impact of extremes of risk-taking preference on surgical decision making can be an important part of decreasing adverse events during LC and should be evaluated. (J Am Coll Surg 2009;209:17-24. (C) 2009 by the American College of Surgeons)

BACKGROUND: The risk of common bile duct injury during laparoscopic cholecystectomy (LC) is 50% to 70% lower when an intraoperative cholangiograrn (IOC) is used, and this effect is exaggerated among less experienced surgeons. Routine IOC is not universal, and barriers to Its use, including surgeon knowledge, behavior, and attitudes, should be understood in developing quality-improvement interventions aimed at increasing IOC use.STUDY DESIGN: There were 4, 100 general surgeons randomly selected from the American College Of Surgeons who were mailed a Survey about IOC. Surveys with a valid exclusion (retired, no LC experience) were considered responsive but were excluded from data analysis.RESULTS: Forty-four percent responded, with 1,417 surveys analyzed (mean age 51.8 +/- 9.6 years; 89.2% men; 55.3% private practice). Twenty-seven percent of respondents defined themselves as routine IOC users and 91.3% of routine users reported IOC use in more than 75% of LCs performed. Academic surgeons were less often routine users compared with nonacademics (15% versus 30%; p < 0.001). Selective users were more often low-volume (less than 20 LC/year) surgeons (8% versus 15%) as compared with routine users, who were more often high-volume (more than 100 LC/year) surgeons (27% versus 20%). Routine users had more favorable and accurate opinions about IOC (less costly and more protective of injury) than did selective users. Thirty-nine percent of routine users thought IOC decreased the risk of common bile duct injury by at least half compared with 10% of selective users.CONCLUSIONS: Surgeons at greatest risk for causing common bile duct injury (inexperienced, low-volume Surgeons) and those who have the greatest opportunity to train others are less likely to use IOC routinely. These represent target groups for quality-improvement interventions aimed at broader IOC use. (J Am Coll Surg 2008;207:821-830. (C) 2008 by the American College Of Surgeons)

Background Laparoscopic adjustable gastric banding (LAGB) and laparoscopic Roux-en-Y gastric bypass (LRYGB) are the two most commonly performed bariatric procedures. Although both procedures likely reduce healthcare expenditures related to the resolution of co-morbid conditions, they have different rates of perioperative risks and different rates of associated weight loss. We designed a model to evaluate the incremental cost-effectiveness of these procedures compared with nonoperative weight loss interventions and with each other. Methods We used a deterministic, payer-perspective model comparing the lifetime expected costs and outcomes of LAGB, LRYGB, and nonoperative treatment. The major endpoints were survival, health-related quality of life, and weight loss. Life expectancy and lifetime medical costs were calculated across age, gender, and body mass index (BMI) strata using previously published data. Results For both men and women, LRYGB and LAGB were cost-effective at <$25,000/quality-adjusted life-year (QALY) even when evaluating the full range of baseline BMI and estimates of adverse outcomes, weight loss, and costs. For base-case scenarios in men (age 35 y, BMI 40 kg/m2), the incremental cost-effectiveness was $11,604/QALY for LAGB compared with $18,543/QALY for LRYGB. For base-case scenarios in women (age 35 y, BMI 40 kg/m2), the incremental cost-effectiveness was $8878/QALY for LAGB compared with $14,680/QALY for LRYGB. Conclusion The modeled cost-effectiveness analysis showed that both operative interventions for morbid obesity, LAGB and RYGB, were cost-effective at <$25,000 and that LAGB was more cost-effective than RYGB for all base-case scenarios.

Rodents have been used to examine physiologic changes after bariatric surgery, but differences in gastric/vagal anatomy may limit their utility. Swine may be a more appropriate animal model because of anatomic and physiologic similarities to humans.The aim of this study was to establish a survival model of Roux-en-Y gastric bypass (RYGBP) in swine and to evaluate its potential in studies of physiology.