Objective: The United States Office of the National Coordinator for Health Information Technology sponsored the development of a “high-priority” list of drug-drug interactions (DDIs) to be used for clinical decision support. We assessed current adoption of this list and current alerting practice for these DDIs with regard to alert implementation (presence or absence of an alert) and display (alert appearance as interruptive or passive). Materials and methods: We conducted evaluations of electronic health records (EHRs) at a convenience sample of health care organizations across the United States using a standardized testing protocol with simulated orders. Results: Evaluations of 19 systems were conducted at 13 sites using 14 different EHRs. Across systems, 69% of the high-priority DDI pairs produced alerts. Implementation and display of the DDI alerts tested varied between systems, even when the same EHR vendor was used. Across the drug pairs evaluated, implementation and display of DDI alerts differed, ranging from 27% (4/15) to 93% (14/15) implementation. Discussion: Currently, there is no standard of care covering which DDI alerts to implement or how to display them to providers. Opportunities to improve DDI alerting include using differential displays based on DDI severity, establishing improved lists of clinically significant DDIs, and thoroughly reviewing organizational implementation decisions regarding DDIs. Conclusion: DDI alerting is clinically important but not standardized. There is significant room for improvement and standardization around evidence-based DDIs.

Anesthesia information management systems (AIMS) are sophisticated hardware and software technology solutions that can provide electronic feedback to anesthesia providers. This feedback can be tailored to provide clinical decision support (CDS) to aid clinicians with patient care processes, documentation compliance, and resource utilization. We conducted a systematic review of peer-reviewed articles on near real-time and point-of-care CDS within AIMS using the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols. Studies were identified by searches of the electronic databases Medline and EMBASE. Two reviewers screened studies based on title, abstract, and full text. Studies that were similar in intervention and desired outcome were grouped into CDS categories. Three reviewers graded the evidence within each category. The final analysis included 25 articles on CDS as implemented within AIMS. CDS categories included perioperative antibiotic prophylaxis, post-operative nausea and vomiting prophylaxis, vital sign monitors and alarms, glucose management, blood pressure management, ventilator management, clinical documentation, and resource utilization. Of these categories, the reviewers graded perioperative antibiotic prophylaxis and clinical documentation as having strong evidence per the peer reviewed literature. There is strong evidence for the inclusion of near real-time and point-of-care CDS in AIMS to enhance compliance with perioperative antibiotic prophylaxis and clinical documentation. Additional research is needed in many other areas of AIMS-based CDS.

An action-oriented alerts taxonomy according to structure, actions, and implicit intended process outcomes using a set of 333 rule-based alerts at Kaiser Permanente Northwest (KPNW) was developed. The authors identified 9 major and 17 overall classes of alerts and developed a specific metric approach for 5 of these classes, including the 3 most numerous ones in KPNW, accounting for 224 (67%) of the alerts.

OBJECTIVE:To assess problem list completeness using an objective measure across a range of sites, and to identify success factors for problem list completeness.METHODS:We conducted a retrospective analysis of electronic health record data and interviews at ten healthcare organizations within the United States, United Kingdom, and Argentina who use a variety of electronic health record systems: four self-developed and six commercial. At each site, we assessed the proportion of patients who have diabetes recorded on their problem list out of all patients with a hemoglobin A1c elevation>=7.0%, which is diagnostic of diabetes. We then conducted interviews with informatics leaders at the four highest performing sites to determine factors associated with success. Finally, we surveyed all the sites about common practices implemented at the top performing sites to determine whether there was an association between problem list management practices and problem list completeness.RESULTS:Problem list completeness across the ten sites ranged from 60.2% to 99.4%, with a mean of 78.2%. Financial incentives, problem-oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture were identified as success factors at the four hospitals with problem list completeness at or near 90.0%.DISCUSSION:Incomplete problem lists represent a global data integrity problem that could compromise quality of care and put patients at risk. There was a wide range of problem list completeness across the healthcare facilities. Nevertheless, some facilities have achieved high levels of problem list completeness, and it is important to better understand the factors that contribute to success to improve patient safety.CONCLUSION:Problem list completeness varies substantially across healthcare facilities. In our review of EHR systems at ten healthcare facilities, we identified six success factors which may be useful for healthcare organizations seeking to improve the quality of their problem list documentation: financial incentives, problem oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture.

As we enter the realm in which many laboratory and quantitative parameters can be accumulated about a patient, and where natural language processing methods are becoming more capable of extracting and codifying data from unstructured narrative reports, the matching of patients to smaller subgroups is beginning to be more broadly feasible. In the future, population-based decision support is likely to be much more patient-specific and much more reliant on predictive models for specific subpopulations. This chapter reviews the current state and trends in applying clinical decision support to population management and population-derived data to decision support for the individual patient.

The promise of Clinical Decision Support (CDS) has always been to transform patient care and improve patient safety with delivery of timely and appropriate recommendations that are both patient-specific and more often than not appropriately actionable. However, the users of CDS, providers, are frequently bombarded with inappropriate and inapplicable CDS that is frequently neither informational, integrated into the workflow, patient-specific, and may present out of date and irrelevant recommendations. The life cycle of Clinical Decision Support begins with a request for CDS, continues with design and implementation, and concludes with ongoing knowledge maintenance. This State of the Practice will look at how using the best science and latest knowledge regarding CDS can create request and maintenance processes that work in the real world. Dr. David Bates will present the best science and knowledge behind CDS that works. Dr.’s Joseph Kannry and Thomas Yackel will present case studies of CDS requests and design processes that use this science to generate useful, useable, and timely patient-specific recommendations. Dr. Tonya Hongsermeier will present best practices in knowledge maintenance. Finally, Dr. Michael Krall will present a case study of knowledge maintenance from Kaiser Permanente that results in appropriate and up-to-date CDS.

BACKGROUND:Many computerized provider order entry (CPOE) systems include the ability to create electronic order sets: collections of clinically related orders grouped by purpose. Order sets promise to make CPOE systems more efficient, improve care quality and increase adherence to evidence-based guidelines. However, the development and implementation of order sets can be expensive and time-consuming and limited literature exists about their utilization.METHODS:Based on analysis of order set usage logs from a diverse purposive sample of seven sites with commercially and internally developed inpatient CPOE systems, we developed an original order set classification system. Order sets were categorized across seven non-mutually exclusive axes: admission/discharge/transfer (ADT), perioperative, condition-specific, task-specific, service-specific, convenience, and personal. In addition, 731 unique subtypes were identified within five axes: four in ADT (S=4), three in perioperative, 144 in condition-specific, 513 in task-specific, and 67 in service-specific.RESULTS:Order sets (n=1914) were used a total of 676,142 times at the participating sites during a one-year period. ADT and perioperative order sets accounted for 27.6% and 24.2% of usage respectively. Peripartum/labor, chest pain/acute coronary syndrome/myocardial infarction and diabetes order sets accounted for 51.6% of condition-specific usage. Insulin, angiography/angioplasty and arthroplasty order sets accounted for 19.4% of task-specific usage. Emergency/trauma, obstetrics/gynecology/labor delivery and anesthesia accounted for 32.4% of service-specific usage. Overall, the top 20% of order sets accounted for 90.1% of all usage. Additional salient patterns are identified and described.CONCLUSION:We observed recurrent patterns in order set usage across multiple sites as well as meaningful variations between sites. Vendors and institutional developers should identify high-value order set types through concrete data analysis in order to optimize the resources devoted to development and implementation.

Representatives from a consortium of academic institutions, healthcare organisations, and commercial entities have collaborated to implement and evaluate a multi–layered representation framework targeted to promote the sharing and distribution of clinical practice guideline knowledge artefacts. Definition of a metadata model for knowledge artefacts submitted to a shared repository based on this framework proceeded through an iterative process that sought to strike a balance between the need to provide enough specification to allow for efficient indexing and retrieval and recognition of the potential burden imposed by excessive requirements.

BACKGROUND:Computer-based clinical decision support (CDS) systems have been shown to improve quality of care and workflow efficiency, and health care reform legislation relies on electronic health records and CDS systems to improve the cost and quality of health care in the United States; however, the heterogeneity of CDS content and infrastructure of CDS systems across sites is not well known. OBJECTIVE:We aimed to determine the scope of CDS content in diabetes care at six sites, assess the capabilities of CDS in use at these sites, characterize the scope of CDS infrastructure at these sites, and determine how the sites use CDS beyond individual patient care in order to identify characteristics of CDS systems and content that have been successfully implemented in diabetes care. METHODS:We compared CDS systems in six collaborating sites of the Clinical Decision Support Consortium. We gathered CDS content on care for patients with diabetes mellitus and surveyed institutions on characteristics of their site, the infrastructure of CDS at these sites, and the capabilities of CDS at these sites. RESULTS:The approach to CDS and the characteristics of CDS content varied among sites. Some commonalities included providing customizability by role or user, applying sophisticated exclusion criteria, and using CDS automatically at the time of decision-making. Many messages were actionable recommendations. Most sites had monitoring rules (e.g. assessing hemoglobin A1c), but few had rules to diagnose diabetes or suggest specific treatments. All sites had numerous prevention rules including reminders for providing eye examinations, influenza vaccines, lipid screenings, nephropathy screenings, and pneumococcal vaccines. CONCLUSION:Computer-based CDS systems vary widely across sites in content and scope, but both institution-created and purchased systems had many similar features and functionality, such as integration of alerts and reminders into the decision-making workflow of the provider and providing messages that are actionable recommendations.

Most computerized physician order entry (CPOE) systems have built-in support for order sets (collections of orders grouped by a clinical purpose). Evidence and experience suggest that order sets are important tools for ordering efficiency and decision support and may influence ordering. Developing and maintaining order sets is costly, so hospitals often must prioritize which order sets can be created. We analyzed order set utilization at seven diverse sites with CPOE. The number of order sets per site ranged from 81 to 535, and the number of order set uses per discharge ranged from 0.48 to 9.89. We also compared the top ten order sets at each site, and found many commonalities, such as generic and condition-specific admission order sets, surgical sets and clinical pathways. We also found that, at each site, utilization of order sets was skewed, with a small number of order sets comprising the bulk of utilization. These findings may be useful for order sets developers, particularly in settings where resources are constrained and the most important order sets must be developed first.

Background Real-time clinical decision support (CDS) integrated into clinicians' workflow has the potential to profoundly affect the cost, quality, and safety of health care delivery. Recent reports have identified a surprisingly low acceptance rate for different types of CDS. We hypothesized that factors affecting CDS system acceptance could be categorized as relating to differences in patients, physicians, CDS-type, or environmental characteristics. Methods We conducted a survey of all adult primary care physicians (PCPs, n = 225) within our group model Health Maintenance Organization (HMO) to identify factors that affect their acceptance of CDS. We defined clinical decision support broadly as "clinical information" that is either provided to you or accessible by you, from the clinical workstation (e.g., enhanced flow sheet displays, health maintenance reminders, alternative medication suggestions, order sets, alerts, and access to any internet-based information resources). Results 110 surveys were returned (49%). There were no differences in the age, gender, or years of service between those who returned the survey and the entire adult PCP population. Overall, clinicians stated that the CDS provided "helps them take better care of their patients" (3.6 on scale of 1:Never – 5:Always), "is worth the time it takes" (3.5), and "reminds them of something they've forgotten" (3.2). There was no difference in the perceived acceptance rate of alerts based on their type (i.e., cost, safety, health maintenance). When asked about specific patient characteristics that would make the clinicians "more", "equally" or "less" likely to accept alerts: 41% stated that they were more (8% stated "less") likely to accept alerts on elderly patients (> 65 yrs); 38% were more (14% stated less) likely to accept alerts on patients with more than 5 current medications; and 38% were more (20% stated less) likely to accept alerts on patients with more than 5 chronic clinical conditions. Interestingly, 80% said they were less likely to accept alerts when they were behind schedule and 84% of clinicians admitted to being at least 20 minutes behind schedule "some", "most", or "all of the time". Conclusion Even though a majority of our clinical decision support suggestions are not explicitly followed, clinicians feel they are of benefit and would be even more beneficial if they had more time available to address them.

Methods: This study used an interrupted time series de- sign. In a health maintenance organization with an elec- tronic medical record, we evaluated the effectiveness of electronic medical record alerts and group academic de- tailing to reduce the coprescribing of warfarin and in- teracting medications. Participants were 239 primary care providers at 15 primary care clinics and 9910 patients taking warfarin. All 15 clinics received electronic medi- cal record alerts for the coprescription of warfarin and 5 interacting medications: acetaminophen, nonsteroi- dal anti-inflammatory medications, fluconazole, metro- nidazole, and sulfamethoxazole. Seven clinics were ran- domly assigned to receive group academic detailing. The primary outcome, the interacting prescription rate (ie, the number of coprescriptions of warfarin-interacting medications per 10 000 warfarin users per month), was analyzed with segmented regression models, control- ling for preintervention trends. Results: At baseline, nearly a third of patients had an interacting prescription. Coinciding with the alerts, there was an immediate and continued reduction in the war- farin-interacting medication prescription rate (from 3294.0 to 2804.2), resulting in a 14.9% relative reduc- tion (95% confidence interval, �19.5 to �10.2) at 12 months. Group academic detailing did not enhance alert effectiveness. Conclusions: This study, using a strong and quasi- experimental design in ambulatory care, found that medi- cation interaction alerts modestly reduced the frequency of coprescribing of interacting medications. Additional ef- forts will be required to further reduce rates of inappro- priate prescribing of warfarin with interacting drugs. Arch Intern Med. 2006;166:1009-1015

Background: Computerized decision support reduces medication errors in inpatients, but limited evidence supports its effectiveness in reducing the coprescribing of interacting medications, especially in the outpatient setting. The usefulness of academic detailing to enhance the effectiveness of medication interaction alerts also is uncertain.Methods: This study used an interrupted time series design. In a health maintenance organization with an electronic medical record, we evaluated the effectiveness of electronic medical record alerts and group academic detailing to reduce the coprescribing of warfarin and interacting medications. Participants were 239 primary care providers at 15 primary care clinics and 9910 patients taking warfarin. All 15 clinics received electronic medical record alerts for the coprescription of warfarin and 5 interacting medications: acetaminophen, nonsteroidal anti-inflammatory medications, fluconazole, metronidazole, and sulfamethoxazole. Seven clinics were randomly assigned to receive group academic detailing. The primary outcome, the interacting prescription rate (ie, the number of coprescriptions of warfarin-interacting medications per 10 000 warfarin users per month), was analyzed with segmented regression models, controlling for preintervention trends.Results: At baseline, nearly a third of patients had an interacting prescription. Coinciding with the alerts, there was an immediate and continued reduction in the war-farin-interacting medication prescription rate ( from 3294.0 to 2804.2), resulting in a 14.9% relative reduction (95% confidence interval, -19.5 to -10.2) at 12 months. Group academic detailing did not enhance alert effectiveness.Conclusions: This study, using a strong and quasi-experimental design in ambulatory care, found that medication interaction alerts modestly reduced the frequency of coprescribing of interacting medications. Additional efforts will be required to further reduce rates of inappropriate prescribing of warfarin with interacting drugs.

Objectives: Computer-based provider order entry (CPOE) systems are implemented to increase both efficiency and accuracy in health care, but these systems often cause a myriad of emotions to arise. This qualitative research investigates the emotions surrounding CPOE implementation and use.Methods: We performed a secondary analysis of several previously collected qualitative data sets from interviews and observations of over 50 individuals. Three researchers worked in parallel to identify themes that expressed emotional responses to CPOE. We then reviewed and classified these quotes using a validated hierarchical taxonomy of semantically homogeneous terms associated with specific emotions.Results: The implementation and use of CPOE systems provoked examples of positive, negative, and neutral emotions. Negative emotional responses were the most prevalent, by far, in all the observations.Conclusion: Designing and implementing CPOE systems is difficult. These systems and the implementation process itself often inspire intense emotions. If designers and implementers fail to recognize that various CPOE features and implementation strategies can increase clinicians' negative emotions, then the systems may fail to become a routine part of the clinical care delivery process. We might alleviate some of these problems by designing positive feedback mechanisms for both the systems and the organizations.

I can't imagine that it will ever be easy to care for multiproblem, chronically ill patients; I suspect that such patients will always be a challenge. However, effective use of a system like KP HealthConnect can help us to be more organized and more focused in this care and can help us in the often very difficult, time-consuming task of reviewing the past status and preparing ourselves for the issues of today. With an up-to-date and managed Problem List, Medication List, and Histories section, we will be able to review in a short time what otherwise would require many minutes. Moreover, performing interim updates is simple and does not require starting from scratch, as often was the case with a paper record. Likewise, the use of SmartLinks will frequently streamline the documentation task. A SmartSet may bring all needed elements together into one easily used package. With KP HealthConnect, we have more electronic record tools, better adapted for specific functions, than ever before. With regular use of these tools, the result will be more time to spend on today's issues, including emotional needs, and on prevention, health maintenance, and chronic disease management. Achieving this goal will require dedication and ongoing effort but should yield a very satisfying and rewarding feeling not unlike that which a master carpenter might experience upon completing a fine china cabinet.

There is very little known about the limits of alerting in the setting of the outpatient Electronic Medical Record (EMR). We are interested in how users value and prefer such alerts. One hundred Kaiser Permanente primary care clinicians were sent a four-page questionnaire. It contained questions related to the usability and usefulness of different approaches to presenting reminder and alert information. The survey also contained questions about the desirability of six categories of alerts. Forty-three of 100 questionnaires were returned. Users generally preferred an active, more intrusive interaction model for "alerts" and a passive, less intrusive model for order messages and other types of reminders and notifications. Drug related alerts were more highly rated than health maintenance or disease state reminders. Users indicated that more alerts would make the system "more useful" but "less easy to use".