OBJECTIVE:Clinical decision support (CDS) can contribute to quality and safety. Prior work has shown that errors in CDS systems are common and can lead to unintended consequences. Many CDS systems use Boolean logic, which can be difficult for CDS analysts to specify accurately. We set out to determine the prevalence of certain types of Boolean logic errors in CDS statements. METHODS:Nine health care organizations extracted Boolean logic statements from their Epic electronic health record (EHR). We developed an open-source software tool, which implemented the Espresso logic minimization algorithm, to identify three classes of logic errors. RESULTS:Participating organizations submitted 260,698 logic statements, of which 44,890 were minimized by Espresso. We found errors in 209 of them. Every participating organization had at least two errors, and all organizations reported that they would act on the feedback. DISCUSSION:An automated algorithm can readily detect specific categories of Boolean CDS logic errors. These errors represent a minority of CDS errors, but very likely require correction to avoid patient safety issues. This process found only a few errors at each site, but the problem appears to be widespread, affecting all participating organizations. CONCLUSION:Both CDS implementers and EHR vendors should consider implementing similar algorithms as part of the CDS authoring process to reduce the number of errors in their CDS interventions.

Objective The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records. Materials and Methods We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices. Results Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: “will monitor or take precautions,” “not clinically significant,” and “benefit outweighs risk.” Discussion We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved. Conclusions Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Objective Assess information integrity (concordance and completeness of documented exam indications from the electronic health record [EHR] imaging order requisition, compared to EHR provider notes), and assess potential impact of indication inaccuracies on exam planning and interpretation. Methods This retrospective study, approved by the Institutional Review Board, was conducted at a tertiary academic medical center. There were 139 MRI lumbar spine (LS-MRI) and 176 CT abdomen/pelvis orders performed 4/1/2016-5/31/2016 randomly selected and reviewed by 4 radiologists for concordance and completeness of relevant exam indications in order requisitions compared to provider notes, and potential impact of indication inaccuracies on exam planning and interpretation. Forty each LS-MRI and CT abdomen/pelvis were re-reviewed to assess kappa agreement. Results Requisition indications were more likely to be incomplete (256/315, 81%) than discordant (133/315, 42%) compared to provider notes (p < 0.0001). Potential impact of discrepancy between clinical information in requisitions and provider notes was higher for radiologist's interpretation than for exam planning (135/315, 43%, vs 25/315, 8%, p < 0.0001). Agreement among radiologists for concordance, completeness, and potential impact was moderate to strong (Kappa 0.66-0.89). Indications in EHR order requisitions are frequently incomplete or discordant compared to physician notes, potentially impacting imaging exam planning, interpretation and accurate diagnosis. Such inaccuracies could also diminish the relevance of clinical decision support alerts if based on information in order requisitions. Conclusions Improved availability of relevant documented clinical information within EHR imaging requisition is necessary for optimal exam planning and interpretation.

Objective: Assess information integrity (concordance and completeness of documented exam indications from the electronic health record [EHR] imaging order requisition, compared to EHR provider notes), and assess potential impact of indication inaccuracies on exam planning and interpretation. Methods: This retrospective study, approved by the Institutional Review Board, was conducted at a tertiary academic medical center. There were 139 MRI lumbar spine (LS-MRI) and 176 CT abdomen/pelvis orders performed 4/1/2016-5/31/2016 randomly selected and reviewed by 4 radiologists for concordance and completeness of relevant exam indications in order requisitions compared to provider notes, and potential impact of indication inaccuracies on exam planning and interpretation. Forty each LS-MRI and CT abdomen/pelvis were re-reviewed to assess kappa agreement. Results: Requisition indications were more likely to be incomplete (256/315, 81%) than discordant (133/315, 42%) compared to provider notes (p<0.0001). Potential impact of discrepancy between clinical information in requisitions and provider notes was higher for radiologist’s interpretation than for exam planning (135/315, 43%, vs 25/315, 8%, p<0.0001). Agreement among radiologists for concordance, completeness, and potential impact was moderate to strong (Kappa 0.66-0.89). Indications in EHR order requisitions are frequently incomplete or discordant compared to physician notes, potentially impacting imaging exam planning, interpretation and accurate diagnosis. Such inaccuracies could also diminish the relevance of clinical decision support alerts if based on information in order requisitions. Conclusions: Improved availability of relevant documented clinical information within EHR imaging requisition is necessary for optimal exam planning and interpretation.

OBJECTIVEPersistent concern exists about the variable and possibly inappropriate utilization of high-cost imaging tests. The purpose of this study is to assess the influence of appropriate use criteria attributes on altering ambulatory imaging orders deemed inappropriate.MATERIALS AND METHODSThis secondary analysis included Medicare Imaging Demonstration data collected from three health care systems in 2011-2013 via the use of clinical decision support (CDS) during ambulatory imaging order entry. The CDS system captured whether orders were inappropriate per the appropriate use criteria of professional societies and provided advice during the intervention period. For orders deemed inappropriate, we assessed the impact of the availability of alternative test recommendations, conflicts with local best practices, and the strength of evidence for appropriate use criteria on the primary outcome of cancellation or modification of inappropriate orders. Expert review determined conflicts with local best practices for 250 recommendations for abdominal and thoracic CT orders. Strength of evidence was assessed for the 15 most commonly triggered recommendations that were deemed inappropriate. A chi-square test was used for univariate analysis.RESULTSA total of 1691 of 63,222 imaging test orders (2.7%) were deemed inappropriate during the intervention period; this amount decreased from 364 of 11,675 test orders (3.1%) in the baseline period (p < 0.00001). Of 270 inappropriate recommendations with alternative test recommendations, 28 (10.4%) were modified, compared with four of 1024 inappropriate recommendations without alternatives (0.4%) (p < 0.0001). Seventy-eight of 250 recommendations (31%) conflicted with local best practices, but only six of 69 inappropriate recommendations (9%) conflicted (p < 0.001). No inappropriate recommendations that conflicted with local best practices were modified. All 15 commonly triggered recommendations had an Oxford Centre for Evidence-Based Medicine level of evidence of 5 (i.e., expert opinion).CONCLUSIONOrders for imaging tests that were deemed inappropriate were modified infrequently, more often with alternative recommendations present and only for appropriate use criteria consistent with local best practices.

Objective: The United States Office of the National Coordinator for Health Information Technology sponsored the development of a “high-priority” list of drug-drug interactions (DDIs) to be used for clinical decision support. We assessed current adoption of this list and current alerting practice for these DDIs with regard to alert implementation (presence or absence of an alert) and display (alert appearance as interruptive or passive). Materials and methods: We conducted evaluations of electronic health records (EHRs) at a convenience sample of health care organizations across the United States using a standardized testing protocol with simulated orders. Results: Evaluations of 19 systems were conducted at 13 sites using 14 different EHRs. Across systems, 69% of the high-priority DDI pairs produced alerts. Implementation and display of the DDI alerts tested varied between systems, even when the same EHR vendor was used. Across the drug pairs evaluated, implementation and display of DDI alerts differed, ranging from 27% (4/15) to 93% (14/15) implementation. Discussion: Currently, there is no standard of care covering which DDI alerts to implement or how to display them to providers. Opportunities to improve DDI alerting include using differential displays based on DDI severity, establishing improved lists of clinically significant DDIs, and thoroughly reviewing organizational implementation decisions regarding DDIs. Conclusion: DDI alerting is clinically important but not standardized. There is significant room for improvement and standardization around evidence-based DDIs.

OBJECTIVE:Increasing the use of generic medications could help control medical costs. However, educational interventions have limited impact on prescriber behavior, and e-prescribing alerts are associated with high override rates and alert fatigue. Our objective was to evaluate the effect of a less intrusive intervention, a redesign of an e-prescribing interface that provides default options intended to "nudge" prescribers towards prescribing generic drugs. METHODS:This retrospective cohort study in an academic ambulatory multispecialty practice assessed the effects of customizing an e-prescribing interface to substitute generic equivalents for brand-name medications during order entry and allow a one-click override to order the brand-name medication. RESULTS:Among drugs with generic equivalents, the proportion of generic drugs prescribed more than doubled after the interface redesign, rising abruptly from 39.7% to 95.9% (a 56.2% increase; 95% confidence interval, 56.0-56.4%; P < .001). Before the redesign, generic drug prescribing rates varied by therapeutic class, with rates as low as 8.6% for genitourinary products and 15.7% for neuromuscular drugs. After the redesign, generic drug prescribing rates for all but four therapeutic classes were above 90%: endocrine drugs, neuromuscular drugs, nutritional products, and miscellaneous products. DISCUSSION:Changing the default option in an e-prescribing interface in an ambulatory care setting was followed by large and sustained increases in the proportion of generic drugs prescribed at the practice. CONCLUSIONS:Default options in health information technology exert a powerful effect on user behavior, an effect that can be leveraged to optimize decision making.

OBJECTIVE. The efficacy of imaging clinical decision support (CDS) varies. Our objective was to identify CDS factors contributing to imaging order cancellation or modification.SUBJECTS AND METHODS. This pre-post study was performed across four institutions participating in the Medicare Imaging Demonstration. The intervention was CDS at order entry for selected outpatient imaging procedures. On the basis of the information entered, computerized alerts indicated to providers whether orders were not covered by guidelines, appropriate, of uncertain appropriateness, or inappropriate according to professional society guidelines. Ordering providers could override or accept CDS. We considered actionable alerts to be those that could generate an immediate order behavior change in the ordering physician (i.e., cancellation of inappropriate orders or modification of orders of uncertain appropriateness that had a recommended alternative). Chi-square and logistic regression identified predictors of order cancellation or modification after an alert.RESULTS. A total of 98,894 radiology orders were entered (83,114 after the intervention). Providers ignored 98.9%, modified 1.1%, and cancelled 0.03% of orders in response to alerts. Actionable alerts had a 10 fold higher rate of modification (8.1% vs 0.7%; p < 0.0001) or cancellation (0.2% vs 0.02%; p < 0.0001) orders compared with nonactionable alerts. Orders from institutions with preexisting imaging CDS had a sevenfold lower rate of cancellation or modification than was seen at sites with newly implemented CDS (1.4% vs 0.2%; p < 0.0001). In multivariate analysis, actionable alerts were 12 times more likely to result in order cancellation or modification. Orders at sites with preexisting CDS were 7.7 times less likely to be cancelled or modified (p < 0.0001).CONCLUSION. Using results from the Medicare Imaging Demonstration project, we identified potential factors that were associated with CDS effect on provider imaging ordering; these findings may have implications for future design of such computerized systems.

OBJECTIVE:To assess problem list completeness using an objective measure across a range of sites, and to identify success factors for problem list completeness.METHODS:We conducted a retrospective analysis of electronic health record data and interviews at ten healthcare organizations within the United States, United Kingdom, and Argentina who use a variety of electronic health record systems: four self-developed and six commercial. At each site, we assessed the proportion of patients who have diabetes recorded on their problem list out of all patients with a hemoglobin A1c elevation>=7.0%, which is diagnostic of diabetes. We then conducted interviews with informatics leaders at the four highest performing sites to determine factors associated with success. Finally, we surveyed all the sites about common practices implemented at the top performing sites to determine whether there was an association between problem list management practices and problem list completeness.RESULTS:Problem list completeness across the ten sites ranged from 60.2% to 99.4%, with a mean of 78.2%. Financial incentives, problem-oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture were identified as success factors at the four hospitals with problem list completeness at or near 90.0%.DISCUSSION:Incomplete problem lists represent a global data integrity problem that could compromise quality of care and put patients at risk. There was a wide range of problem list completeness across the healthcare facilities. Nevertheless, some facilities have achieved high levels of problem list completeness, and it is important to better understand the factors that contribute to success to improve patient safety.CONCLUSION:Problem list completeness varies substantially across healthcare facilities. In our review of EHR systems at ten healthcare facilities, we identified six success factors which may be useful for healthcare organizations seeking to improve the quality of their problem list documentation: financial incentives, problem oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture.

We review several newer modalities to monitor the brain in children with acute neurologic disease in the pediatric intensive care unit, such as partial brain tissue oxygen tension (PbtO2), jugular venous oxygen saturation (SjvO2), near infrared spectroscopy (NIRS), thermal diffusion measurement of cerebral blood flow, cerebral microdialysis, and EEG. We then discuss the informatics challenges to acquire, consolidate, analyze, and display the data. Acquisition includes multiple data types: discrete, waveform, and continuous. Consolidation requires device interoperability and time synchronization. Analysis could include pressure reactivity index and quantitative EEG. Displays should communicate the patient's current status, longitudinal and trend information, and critical alarms.

SummaryObjective: To provide a case report of barriers and promoters to implementing a health information exchange (HIE) tool that supports patient transfers between hospitals and skilled nursing facilities.Methods: A multi-disciplinary team conducted semi-structured telephone and in-person interviews in a purposive sample of HIE organizational informants and providers in New York City who implemented HIE to share patient transfer information. The researchers conducted grounded theory analysis to identify themes of barriers and promoters and took steps to improve the trustworthiness of the results including vetting from a knowledgeable study participant.Results: Between May and October 2011, researchers recruited 18 participants: informaticians, healthcare administrators, software engineers, and providers from a skilled nursing facility. Subjects perceived the HIE tool’s development a success in that it brought together stakeholders who had traditionally not partnered for informatics work, and that they could successfully share patient transfer information between a hospital and a skilled nursing facility. Perceived barriers included lack of hospital stakeholder buy-in and misalignment with clinical workflows that inhibited use of HIE-based patient transfer data. Participants described barriers and promoters in themes related to organizational, technical, and user-oriented issues.The investigation revealed that stakeholders could develop and implement health information technology that technically enables clinicians in both hospitals and skilled nursing facilities to exchange real-time information in support of patient transfers. User level barriers, particularly in the emergency department, should give pause to developers and implementers who plan to use HIE in support of patient transfers.Conclusions: Participants’ experiences demonstrate how stakeholders may succeed in developing and piloting an electronic transfer form that relies on HIE to aggregate, communicate, and display relevant patient transfer data across health care organizations. Their experiences also provide insights for others seeking to develop HIE applications to improve patient transfers between emergency departments and skilled nursing facilities.Citation: Richardson JE, Malhotra S, Kaushal R. A case report in health information exchange for inter-organizational patient transfers. Appl Clin Inf 2014; 5: 642–650http://dx.doi.org/10.4338/ACI-2014-02-CR-0016

BACKGROUND:The federal Electronic Health Record Incentive Program requires electronic reporting of quality from electronic health records, beginning in 2014. Whether electronic reports of quality are accurate is unclear.OBJECTIVE:To measure the accuracy of electronic reporting compared with manual review.DESIGN:Cross-sectional study.SETTING:A federally qualified health center with a commercially available electronic health record.PATIENTS:All adult patients eligible in 2008 for 12 quality measures (using 8 unique denominators) were identified electronically. One hundred fifty patients were randomly sampled per denominator, yielding 1154 unique patients.MEASUREMENTS:Receipt of recommended care, assessed by both electronic reporting and manual review. Sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, and absolute rates of recommended care were measured.RESULTS:Sensitivity of electronic reporting ranged from 46% to 98% per measure. Specificity ranged from 62% to 97%, positive predictive value from 57% to 97%, and negative predictive value from 32% to 99%. Positive likelihood ratios ranged from 2.34 to 24.25 and negative likelihood ratios from 0.02 to 0.61. Differences between electronic reporting and manual review were statistically significant for 3 measures: Electronic reporting underestimated the absolute rate of recommended care for 2 measures (appropriate asthma medication [38% vs. 77%; P < 0.001] and pneumococcal vaccination [27% vs. 48%; P < 0.001]) and overestimated care for 1 measure (cholesterol control in patients with diabetes [57% vs. 37%; P = 0.001]).LIMITATION:This study addresses the accuracy of the measure numerator only.CONCLUSION:Wide measure-by-measure variation in accuracy threatens the validity of electronic reporting. If variation is not addressed, financial incentives intended to reward high quality may not be given to the highest-quality providers.PRIMARY FUNDING SOURCE:Agency for Healthcare Research and Quality.

OBJECTIVE:To be eligible for incentives through the Electronic Health Record (EHR) Incentive Program, many providers using older or locally developed EHRs will be transitioning to new, commercial EHRs. We previously evaluated prescribing errors made by providers in the first year following transition from a locally developed EHR with minimal prescribing clinical decision support (CDS) to a commercial EHR with robust CDS. Following system refinements, we conducted this study to assess the rates and types of errors 2 years after transition and determine the evolution of errors.MATERIALS AND METHODS:We conducted a mixed methods cross-sectional case study of 16 physicians at an academic-affiliated ambulatory clinic from April to June 2010. We utilized standardized prescription and chart review to identify errors. Fourteen providers also participated in interviews.RESULTS:We analyzed 1905 prescriptions. The overall prescribing error rate was 3.8 per 100 prescriptions (95% CI 2.8 to 5.1). Error rates were significantly lower 2 years after transition (p<0.001 compared to pre-implementation, 12 weeks and 1 year after transition). Rates of near misses remained unchanged. Providers positively appreciated most system refinements, particularly reduced alert firing.DISCUSSION:Our study suggests that over time and with system refinements, use of a commercial EHR with advanced CDS can lead to low prescribing error rates, although more serious errors may require targeted interventions to eliminate them. Reducing alert firing frequency appears particularly important. Our results provide support for federal efforts promoting meaningful use of EHRs.CONCLUSIONS:Ongoing error monitoring can allow CDS to be optimally tailored and help achieve maximal safety benefits.CLINICAL TRIALS REGISTRATION:ClinicalTrials.gov, Identifier: NCT00603070.

Purpose: Federal incentives to adopt interoperable, certified electronic health records (EHRs) with electronic prescribing (e-prescribing) are motivating providers using older EHRs to transition to newer EHRs. The objective of this study was to describe, from the perspective of experienced EHR users, the transition from an older, locally developed EHR with minimal clinical decision support (CDS) for e-prescribing to a newer, commercial EHR with more robust CDS for e-prescribing.Methods: This qualitative, case study consisted of observations and semi-structured interviews of adult internal medicine faculty members (n = 19) at an academic-affiliated ambulatory care clinic from January through November 2009. All providers transitioned from the older, locally developed EHR to the newer, commercial EHR in April 2008. We analyzed field notes of observations and transcripts of semi-structured interviews using qualitative methods guided by a grounded theory approach.Results: We identified key themes describing physician experiences. Despite intensive effort by the information systems team to ease the transition, even these experienced e-prescribers found transitioning extremely difficult. The commercial EHR was not perceived as improving medication safety, despite having more robust CDS. Additionally, physicians felt the commercial EHR was too complex, reducing their efficiency.Conclusions: This is among the first studies examining physician experiences transitioning between an older, locally developed EHR to a newer, commercial EHR with more robust CDS for e-prescribing. Understanding physician experiences with this type of transition and their general preferences for prescribing applications may lead to less disruptive system implementations and better designed EHRs that are more readily accepted by providers. In this way, productivity and safety benefits may be maximized while mitigating potential threats associated with transitions. ClinicalTrials.gov, Identifier: NCT00603070. (C) 2012 Elsevier Ireland Ltd. All rights reserved.

Prescription drugs are a significant component of the ever increasing health care costs. We describe the effects on generic medication prescribing behavior achieved through redesign of the order entry interface of our institutions ambulatory electronic health record. The redesign involved custom programming that automatically substituted brand medications with their generic equivalents and only allowed continuation with the brand medication if the clinician made an extra mouse click selecting "dispense as written". We conducted a before-after retrospective study around the time of the redesign and witnessed a net 36.9% percentage increase in the number of generic medications prescribed.