OBJECTIVES:Little is known about patient safety risks in outpatient surgery. Inpatient surgical adverse events (AEs) risk factors include patient- (e.g., advanced age), process- (e.g., inadequate preoperative assessment), or structure-related characteristics (e.g., low surgical volume); however, these factors may differ from outpatient care where surgeries are often elective and in younger/healthier patients. We undertook an exploratory qualitative research project to identify risk factors for AEs in outpatient surgery.METHODS:We developed a conceptual framework of patient, process, and structure factors associated with surgical AEs on the basis of a literature review. This framework informed our semistructured interview guide with (1) open-ended questions about a specific outpatient AE that the participant experienced and (2) outpatient surgical patient safety risk factors in general. We interviewed nationwide Veterans Health Administration surgical staff. Results were coded on the basis of categories in the conceptual framework, and additional themes were identified using content analysis.RESULTS:Fourteen providers representing diverse surgical roles participated. Ten reported witnessing an AE, and everyone provided input on risk factors in our conceptual framework. We did not find evidence that patient race/age, surgical technique, or surgical volume affected patient safety. Emerging factors included patient compliance, postoperative patient assessments/instruction, operating room equipment needs, and safety culture.CONCLUSIONS:Surgical staff are familiar with AEs and patient safety problems in outpatient surgery. Our results show that processes of care undertaken by surgical providers, as opposed to immutable patient characteristics, may affect the occurrence of AEs. The factors we identified may facilitate more targeted research on outpatient surgical AEs.

ObjectiveDevelop and validate a surveillance model to identify outpatient surgical adverse events (AEs) based on previously developed electronic triggers.Data SourcesVeterans Health Administration's Corporate Data Warehouse.Study DesignSix surgical AE triggers, including postoperative emergency room visits and hospitalizations, were applied to FY2012–2014 outpatient surgeries (n = 744,355). We randomly sampled trigger‐flagged and unflagged cases for nurse chart review to document AEs and measured positive predictive value (PPV) for triggers. Next, we used chart review data to iteratively estimate multilevel logistic regression models to predict the probability of an AE, starting with the six triggers and adding in patient, procedure, and facility characteristics to improve model fit. We validated the final model by applying the coefficients to FY2015 outpatient surgery data (n = 256,690) and reviewing charts for cases at high and moderate probability of an AE.Principal FindingsOf 1,730 FY2012–2014 reviewed surgeries, 350 had an AE (20 percent). The final surveillance model c‐statistic was 0.81. In FY2015 surgeries with >0.8 predicted probability of an AE (n = 405, 0.15 percent), PPV was 85 percent; in surgeries with a 0.4–0.5 predicted probability of an AE, PPV was 38 percent.ConclusionsThe surveillance model performed well, accurately identifying outpatient surgeries with a high probability of an AE.

Background: Research on adverse events (AEs) in outpatient surgery has been limited. As part of a Veterans Health Administration (VA) project on AE surveillance, we chart-reviewed selected outpatient surgical cases to characterize the nature and severity of AEs. Methods: We abstracted financial year 2012-2015 VA outpatient surgery cases selected with high (n = 1185) and low (n = 1072) likelihood of an AE based on postoperative health care utilization. The abstraction tool included established AE definitions and validated harm and severity scales. Results: We found AEs in 608 high-likelihood (51%) and 126 low-likelihood outpatient surgical procedures (12%). Among 1010 unique AEs, the most common were wound issues (n = 261, 26%), urinary retention (23%), and urinary tract infections (12%). While 63% of all AEs involved minimal harm, 28% required hospitalization, and 9% were severely harmful including 8 AEs requiring intervention to sustain life and 2 deaths. Overall, 102 AEs (10%) required, at minimum, a repeat surgery to treat. Conclusions: Among VA outpatient surgical procedures selected based on likelihood of an AE, nearly 40% of identified events carried more than minimal patient harm, undermining the claim that outpatient surgery is relatively safe. Prevalent and preventable AEs such as wound dehiscence and urinary retention may be useful targets for quality improvement.

Background: Given that patient safety measures are increasingly used for public reporting and pay-for performance, it is important for stakeholders to understand how to use these measures for improvement. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are one particularly visible set of measures that are now used primarily for public reporting and pay-for-performance among both private sector and Veterans Health Administration (VA) hospitals. This trend generates a strong need for stakeholders to understand how to interpret and use the PSIs for quality improvement (QI). The goal of this study was to develop an educational program and tailor it to stakeholders' needs. In this paper, we share what we learned from this program development process. Methods: Our study population included key VA stakeholders involved in reviewing performance reports and prioritizing and initiating quality/safety initiatives. A pre-program formative evaluation through telephone interviews and web-based surveys assessed stakeholders' educational needs/interests. Findings from the formative evaluation led to development and implementation of a cyberseminar-based program, which we tailored to stakeholders' needs/interests. A post-program survey evaluated program participants' perceptions about the PSI educational program. Results: Interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Survey results informed us on what program delivery mode and content topics were of high interest. Six cyberseminars were developed-three of which focused on two content areas that were noted of greatest interest: learning how to use PSIs for monitoring trends and understanding how to interpret PSIs. We also used snapshots of VA PSI reports so that participants could directly apply learnings. Although initial interest in the program was high, actual attendance was low. However, post-program survey results indicated that perceptions about the program were positive. Conclusions: Conducting a formative evaluation was a highly important process in program development. The useful information that we collected through the interviews and surveys allowed us to tailor the program to stakeholders' needs and interests. Our experiences, particularly with the formative evaluation process, yielded valuable lessons that can guide others when developing and implementing similar educational programs.

Purpose Given the pace of industry change and the rapid diffusion of high reliability organization (HRO) approaches, lags and divergences have arisen between research and practice in healthcare. The purpose of this paper is to explore several of these theory-practice gaps and propose implications for research and practice. Design/methodology/approach Classic and cutting-edge HRO literature is applied to analyze two industry trends: delivery system integration, and the confluence of patient-as-consumer and patient-centered care. Findings Highly reliable integrated delivery systems will likely function very differently from classic HRO organizations. Both practitioners and researchers should address conditions such as how a system is bounded, how reliable the system should be and how interdependencies are handled. Additionally, systems should evaluate the added uncertainty and variability introduced by enhanced agency on the part of patients/families in decision making and in processes of care. Research limitations/implications Dramatic changes in the sociotechnical environment are influencing the coupling and interactivity of system elements in healthcare. Researchers must address the maintenance of reliability across organizations and the migration of decision-making power toward patients and families. Practical implications As healthcare systems integrate, managers attempting to apply HRO principles must recognize how these systems present new and different reliability-related challenges and opportunities. Originality/value This paper provides a starting point for the advancement of research and practice in high-reliability healthcare by providing an in-depth exploration of the implications of two major industry trends.

BACKGROUND:Adverse event (AE) rates in outpatient surgery are inconsistently reported, partly because of the lack of a standard definition of outpatient surgery. We compared the types and rates of surgical procedures defined by two national healthcare agencies: Health Care Cost Institute (HCCI) and the Healthcare Cost and Utilization Project (HCUP) and considered implications for quality measurement. METHODS:We used HCCI and HCUP definitions to identify FY2012-14 VA outpatient surgeries. RESULTS:There were six times as many HCCI surgeries as HCUP (6,575,830 versus 1,086,640). Ninety-nine percent of HCUP-defined surgeries were also identified by HCCI. More HCUP surgeries had higher average Medicare Relative Value Units then HCCI surgeries [5.3 (SD = 4.4) versus 1.6 (SD = 2.3) RVUs]. CONCLUSIONS:Rates and types of procedures vary widely between definitions. Quality measurement using HCCI versus HCUP may produce significantly lower AE rates because many of the surgeries included reflect low complexity and potentially low risk of AEs.

Health care systems are increasingly burdened by the large numbers of safety measures currently being reported. Within the Veterans Administration (VA), most safety reporting occurs within organizational silos, with little involvement by the frontline users of these measures. To provide a more integrated picture of patient safety, the study team partnered with multiple VA stakeholders and engaged potential frontline users at 2 hospitals to develop a Guiding Patient Safety (GPS) tool. The GPS is currently in its fourth generation; once approval is obtained from senior leadership, implementation will begin. Stakeholders were enthusiastic about the GPS's user-friendly format, comprehensive content, and potential utility for improving safety. These findings suggest that stakeholder engagement is a critical first step in the development of tools that will more likely be used by frontline users. Policy makers and researchers may consider adopting this innovative partnered-research model in developing future national initiatives to deliver meaningful programs to frontline users.

Background— A standard definition of outpatient surgery that is aligned with the perspective of surgical providers is needed for consistent and focused surgical quality measurement. We sought to clarify the procedure characteristics that define outpatient surgery through a consensus process with multidisciplinary surgical staff from the Veterans Health Administration (VA). Methods— A convenience sample of 14 VA surgical staff participated in three rounds of a modified-Delphi process. They rated procedure characteristics (e.g., settings, providers, anesthesia, and incision type) to include in a definition of outpatient surgery for adverse event detection. Results— Consensus was reached on 63% of the criteria. Participants agreed to exclude procedures performed by gastroenterologists; in contrast, they agreed to retain endoscopic procedures. There was inter and intra-variation between rounds but no pattern based on staff discipline. Conclusions— Contradictory responses and the lack of consensus on several key questions revealed few procedure characteristics that surgical staff agreed could be used to define outpatient surgery. Our findings suggest that contextual factors associated with specific procedures may be important; future efforts to define outpatient surgery should consider both the characteristics of the procedure and the procedure itself.

Background: The lack of a tool for categorizing and differentiating hospitals according to their high reliability organization (HRO)-related characteristics has hindered progress toward implementing and sustaining evidence-based HRO practices. Hospitals would benefit both from an understanding of the organizational characteristics that support HRO practices and from knowledge about the steps necessary to achieve HRO status to reduce the risk of harm and improve outcomes. The High Reliability Health Care Maturity (HRHCM) model, a model for health care organizations' achievement of high reliability with zero patient harm, incorporates three major domains critical for promoting HROs-Leadership, Safety Culture, and Robust Process Improvement (R). A study was conducted to examine the content validity of the HRHCM model and evaluate whether it can differentiate hospitals' maturity levels for each of the model's components.Methods: Staff perceptions of patient safety at six US Department of Veterans Affairs (VA) hospitals were examined to determine whether all 14 HRHCM components were present and to characterize each hospital's level of organizational maturity.Results: Twelve of the 14 components from the HRHCM model were detected; two additional characteristics emerged that are present in the HRO literature but not represented in the model-teamwork culture and system-focused tools for learning and improvement. Each hospital's level of organizational maturity could be characterized for 9 of the 14 components.Discussion: The findings suggest the HRHCM model has good content validity and that there is differentiation between hospitals on model components. Additional research is needed to understand how these components can be used to build the infrastructure necessary for reaching high-reliability.

BACKGROUND:Adverse drug event (ADE) detection is an important priority for patient safety research. Trigger tools have been developed to help identify ADEs. In previous work we developed seven concurrent, action-oriented, electronic trigger algorithms designed to prompt clinicians to address ADEs in outpatient care.OBJECTIVES:We assessed the potential adoption and usefulness of the seven triggers by testing the positive predictive validity and obtaining stakeholder input.METHODS:We adapted ADE triggers, "bone marrow toxin-white blood cell count (BMT-WBC)," "bone marrow toxin - platelet (BMT-platelet)," "potassium raisers," "potassium reducers," "creatinine," "warfarin," and "sedative hypnotics," with logic to suppress flagging events with evidence of clinical intervention and applied the triggers to 50,145 patients from three large health care systems. Four pharmacists assessed trigger positive predictive value (PPV) with respect to ADE detection (conservatively excluding ADEs occurring during clinically appropriate care) and clinical usefulness (i.e., whether the trigger alert could change care to prevent harm). We measured agreement between raters using the free kappa and assessed positive PPV for the trigger's detection of harm, clinical usefulness, and both. Stakeholders from the participating health care systems rated the likelihood of trigger adoption and the perceived ease of implementation.FINDINGS:Agreement between pharmacist raters was moderately high for each ADE trigger (kappa free > 0.60). Trigger PPVs for harm ranged from 0 (Creatinine, BMT-WBC) to 17 percent (potassium raisers), while PPV for care change ranged from 0 (WBC) to 60 percent (Creatinine). Fifteen stakeholders rated the triggers. Our assessment identified five of the seven triggers as good candidates for implementation: Creatinine, BMT-Platelet, Potassium Raisers, Potassium Reducers, and Warfarin.CONCLUSIONS:At least five outpatient ADE triggers performed well and merit further evaluation in outpatient clinical care. When used in real time, these triggers may promote care changes to ameliorate patient harm.

Despite promulgation of evidence-based patient care practices, improvement in outcomes has been slow. Successful adoption of better practices may be hindered by organizational contexts. Organizational culture is one important contextual factor. Given the importance of organizational controls, such as standardized practices, for quality performance, the control-oriented aspects of culture may be particularly important. However, evidence is ambiguous as to the culture-outcome relationship in healthcare. Given the role of employee engagement as a moderator of other relationships between organizational context and performance, we hypothesize that organizational conditions for employee engagement moderate the culture-outcome relationship. We tested our hypotheses in 101 Veterans Health Administration hospitals, using employee surveys to measure two types of control-oriented culture (bureaucratic and production cultures) and three conditions for engagement (meaningfulness, psychological safety and resources). Our outcome was 30-day all-cause mortality. We found that bureaucratic culture reduced mortality, production culture increased mortality, and all three conditions for engagement moderated both culture-outcome relationships. Conditions for engagement diminished both the favorable effects of bureaucratic culture and the unfavorable effects of production culture. We conclude with implications for research and practice, including the suggestion that improvement-focused healthcare managers attend to interactions among factors such as organizational culture and employee engagement.

Increasing use of Agency for Healthcare Research and Quality's Patient Safety Indicators (PSIs) for hospital performance measurement intensifies the need to critically assess their validity. Our study examined the extent to which variation in PSI composite score is related to differences in hospital organizational structures or processes (i.e., criterion validity). In site visits to three Veterans Health Administration hospitals with high and three with low PSI composite scores ("low performers" and "high performers," respectively), we interviewed a cross-section of hospital staff. We then coded interview transcripts for evidence in 13 safety-related domains and assessed variation across high and low performers. Evidence of leadership and coordination of work/communication (organizational process domains) was predominantly favorable for high performers only. Evidence in the other domains was either mixed, or there were insufficient data to rate the domains. While we found some evidence of criterion validity, the extent to which variation in PSI rates is related to differences in hospitals' organizational structures/processes needs further study.

Background: Leveraging Frontline Expertise (LFLE) is a patient safety intervention for engaging senior managers with the work-systems challenges faced by frontline workers and ensuring follow-up and accountability for systemic change. A study was conducted to assess the ability to refine, implement, and demonstrate the effectiveness of LFLE, which was designed for and tested in private-sector hospitals, in a Department of Veterans Affairs medical center (VAMC), typically a more hierarchical setting.Methods: LFLE was pilot tested in an urban, East coast-based VAMC, which implemented LFLE in its emergency department and operating room, with the medical/surgical ward and ICU serving as controls. A 20-month multi-method evaluation involved interviews, observation, data-tracking forms, and surveys to measure participant perceptions of the program, operational benchmarks of effectiveness, and longitudinal change in safety climate.Results: Implementation showed fidelity to program design. Participating units identified 22 improvement opportunities, 16 (73%) of which were fully or partially resolved. Senior managers' attitudes toward LFLE were more positive than those of frontline staff, whose attitudes were mixed. Perceptions of safety climate deteriorated during the study period in the implementation units relative to controls.Discussion: LFLE can be implemented in the VA, yield work-system improvements, and increase alignment of improvement aims and actions across hierarchical levels. Yet the results also warn against dangers inherent in adapting improvement programs to new settings. Findings suggest the need for active listening and learning from frontline staff by senior managers and trust building across hierarchical levels.

Both leadership support and staff involvement are essential to healthcare improvement. Leaders of healthcare delivery organizations need practical, reliable ways to engage with frontline staff such that improvement effectiveness is enhanced. Leveraging Frontline Expertise (LFLE) is a structured program for engaging senior managers with the work-systems challenges faced by frontline workers and ensuring follow-up and accountability for systemic change. It seeks to improve quality and safety directly through action to correct problems and indirectly through culture change by demonstrating the institution’s commitment to quality and safety in ways that are meaningful to employees. Designed for and tested in private sector hospitals, this study sought to assess our ability to refine, implement, and demonstrate the effectiveness of LFLE in a Veterans Administration Medical Center (VAMC), typically a more hierarchical setting, and to identify factors that could enhance improvement-related leadership engagement activities with frontline staff. We pilot-tested LFLE in an urban, east coast-based VAMC, which implemented LFLE in its Emergency Department and Operating Room, using its Medical-Surgical ward and Intensive Care Unit as controls. We conducted a multi-method evaluation over 18 months, involving interviews, observation, specialized data-tracking forms, and surveys to measure participant perceptions of the program, operational benchmarks of effectiveness, and longitudinal change in safety climate. Implementation showed fidelity to program design. Participating units identified 22 improvement opportunities and fully or partially resolved 73% of these. Executives’ attitudes towards LFLE were favorable. Frontline staff attitudes were mixed, and perceptions of safety climate deteriorated during the study period in the two implementation-units relative to controls. Results suggest LFLE can be implemented in the VA, yield work-system improvements, and increase alignment of improvement aims and actions across hierarchical levels; they also warn against dangers inherent in adapting improvement programs to new settings. This case study also demonstrates how more hierarchical settings like the VA can be both enabling and constraining for improvement programs such as LFLE. Findings suggest factors that may enhance effectiveness of improvement-related executive engagement with frontline staff include active listening to and learning from frontline staff by senior managers and trust building across hierarchical levels. Ultimately, understanding and appreciation for program objectives is likely as important for their accomplishment as strict adherence to prescribed processes.

BACKGROUND:Improving quality and patient safety is of increasing strategic importance to health care organizations. However, simply increasing the volume of quality improvement (QI) activity does not necessarily improve patient outcomes. There is a need for greater understanding of QI success factors.PURPOSE:This study looked for differences in QI implementation across hospitals with a range of performance on Patient Safety Indicators.METHODS:We conducted an exploratory comparative case study of 4 Veterans Health Administration hospitals including site visits and interviews with leaders and staff.FINDINGS:Two themes emerged. Project-level QI learning is assessing and modifying specific QI projects relative to expectations. Program-level QI learning is assessing and modifying the overall QI endeavor. The nature of project-level QI learning was similar across sites, whereas we identified qualitative differences across organizations in program-level QI learning. The highest performing organization was evaluating and refining its overall approach to QI, whereas the others were learning how to build and control QI programs.PRACTICE IMPLICATIONS:Program-level QI learning may be key if a QI program is to succeed in improving patient outcomes. This type of organizational learning entails a big-picture, organization-wide view of QI. It also entails second-order organizational learning based on assessment not only of whether QI is being done correctly but also whether the right QI activities are being done, for the right reasons. The organization is "learning to learn." In addition to gaining mastery and control of QI, leaders regularly engage with staff in rethinking QI and experimenting with new approaches. Leaders also assess how QI activity fits in the organization's developmental journey and how it supports realization of strategy.

Objectives: Adverse drug event (ADE) detection is an important priority of patient safety research. Trigger tools have been developed to help identify ADEs. As part of a larger study, we developed complex and specific trigger algorithms intended for concurrent use with clinical care to detect outpatient ADEs. This article assesses the use of a modified Delphi process to obtain expert consensus on the value of these triggers. Methods: We selected a panel of distinguished clinical and research experts to participate in the modified Delphi process. We created a set of outpatient ADE triggers based on literature review, clinical input, and methodological expertise. The importance of the targeted ADEs, associated drug classes, and trigger logic was used to rate each trigger. Specific criteria were developed to establish consensus. Results: The modified Delphi process established consensus on 6 outpatient ADE triggers to test with patient-level data based on high ratings of utility for patient-level interventions. These triggers focused on detecting ADEs caused by the following drugs or drug classes: bone marrow toxins, potassium raisers, potassium reducers, creatinine, warfarin, and sedative hypnotics. Participants reported including all aspects of the trigger in their ratings, despite our efforts to separate evaluation of clinical need and trigger logic. Participants' expertise affected the evaluation of trigger rules, leading to contradictory feedback on how to improve trigger design. Conclusions: The efficiency of the modified Delphi method could be improved by allowing participants to produce an overall summary score that incorporates both the clinical value and the general logic of the trigger. Revising and improving trigger design should be conducted in a separate process limited only to trigger experts.

This study tested the association between hospital structural characteristics-teaching status, bedsize, and nurse staffing-and potentially preventable adverse events. The authors calculated 14 Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) and a PSI composite, using discharge databases from VA and nonfederal hospitals. This study compared the likelihood of PSI events in hospitals, controlling for structural and other characteristics, including patients' case-mix. Additional controls were employed to account for differences in VA versus nonfederal patients and data. The study found some associations, most notably a positive (unfavorable) association between status as a major teaching hospital and six PSIs. However, for most PSIs, the authors found no association between the structural characteristics tested and likelihood of PSI events. The study's findings extend previous research showing a lack of consistent relationship between structural characteristics and patient safety. However, the results also suggest continued need for examination of the relationship between teaching status and potentially preventable adverse events.

Background The trigger tool methodology uses clinical algorithms applied electronically to 'flag' medical records where adverse events (AEs) have most likely occurred. The authors sought to create surgical triggers to detect AEs in the ambulatory care setting.Methods Four consecutive steps were used to develop ambulatory surgery triggers. First, the authors conducted a comprehensive literature review for surgical triggers. Second, a series of multidisciplinary focus groups (physicians, nurses, pharmacists and information technology specialists) provided user input on trigger selection. Third, a clinical advisory panel designed an initial set of 10 triggers. Finally, a three-phase Delphi process (surgical and trigger tool experts) evaluated and rated the suggested triggers.Results The authors designed an initial set of 10 surgical triggers including five global triggers (flagging medical records for the suspicion of any AE) and five AE-specific triggers (flagging medical records for the suspicion of specific AEs). Based on the Delphi rating of the trigger's utility for system-level interventions, the final triggers were: (1) emergency room visit(s) within 21 days from surgery; (2) unscheduled readmission within 30 days from surgery; (3) unscheduled procedure (interventional radiological, urological, dental, cardiac or gastroenterological) or reoperation within 30 days from surgery; (4) unplanned initial hospital length of stay more than 24 h; and (5) lower-extremity Doppler ultrasound order entry and ICD code for deep vein thrombosis or pulmonary embolus within 30 days from surgery.Conclusion The authors therefore propose a systematic methodology to develop trigger tools that takes into consideration previously published work, end-user preferences and expert opinion.

OBJECTIVES:To examine the criterion validity of the Agency for Health Care Research and Quality (AHRQ) Patient Safety Indicators (PSIs) using clinical data from the Veterans Health Administration (VA) National Surgical Quality Improvement Program (NSQIP).DATA SOURCES:Fifty five thousand seven hundred and fifty two matched hospitalizations from 2001 VA inpatient surgical discharge data and NSQIP chart-abstracted data.STUDY DESIGN:We examined the sensitivities, specificities, positive predictive values (PPVs), and positive likelihood ratios of five surgical PSIs that corresponded to NSQIP adverse events. We created and tested alternative definitions of each PSI.DATA COLLECTION:FY01 inpatient discharge data were merged with 2001 NSQIP data abstracted from medical records for major noncardiac surgeries.PRINCIPAL FINDINGS:Sensitivities were 19-56 percent for original PSI definitions; and 37-63 percent using alternative PSI definitions. PPVs were 22-74 percent and did not improve with modifications. Positive likelihood ratios were 65-524 using original definitions, and 64-744 using alternative definitions. "Postoperative respiratory failure" and "postoperative wound dehiscence" exhibited significant increases in sensitivity after modifications.CONCLUSIONS:PSI sensitivities and PPVs were moderate. For three of the five PSIs, AHRQ has incorporated our alternative, higher sensitivity definitions into current PSI algorithms. Further validation should be considered before most of the PSIs evaluated herein are used to publicly compare or reward hospital performance.