Objective:We describe and evaluate the mapping of computerized tomography (CT) terms from 40 hospitals participating in a health information exchange (HIE) to a standard terminology. Methods:Proprietary CT exam terms and corresponding exam frequency data were obtained from 40 participant HIE sites that transmitted radiology data to the HIE from January 2013 through October 2015. These terms were mapped to the Logical Observations Identifiers Names and Codes (LOINC®) terminology using the Regenstrief LOINC mapping assistant (RELMA) beginning in January 2016. Terms without initial LOINC match were submitted to LOINC as new term requests on an ongoing basis. After new LOINC terms were created, proprietary terms without an initial match were reviewed and mapped to these new LOINC terms where appropriate. Content type and token coverage were calculated for the LOINC version at the time of initial mapping (v2.54) and for the most recently released version at the time of our analysis (v2.63). Descriptive analysis was performed to assess for significant differences in content-dependent coverage between the 2 versions. Results:LOINC's content type and token coverages of HIE CT exam terms for version 2.54 were 83% and 95%, respectively. Two-hundred-fifteen new LOINC CT terms were created in the interval between the releases of version 2.54 and 2.63, and content type and token coverages, respectively, increased to 93% and 99% (P < .001). Conclusion:LOINC's content type coverage of proprietary CT terms across 40 HIE sites was 83% but improved significantly to 93% following new term creation.

BACKGROUND:A health information exchange (HIE)-based prior computed tomography (CT) alerting system may reduce avoidable CT imaging by notifying ordering clinicians of prior relevant studies when a study is ordered. For maximal effectiveness, a system would alert not only for prior same CTs (exams mapped to the same code from an exam name terminology) but also for similar CTs (exams mapped to different exam name terminology codes but in the same anatomic region) and anatomically proximate CTs (exams in adjacent anatomic regions). Notification of previous same studies across an HIE requires mapping of local site CT codes to a standard terminology for exam names (such as Logical Observation Identifiers Names and Codes [LOINC]) to show that two studies with different local codes and descriptions are equivalent. Notifying of prior similar or proximate CTs requires an additional mapping of exam codes to anatomic regions, ideally coded by an anatomic terminology. Several anatomic terminologies exist, but no prior studies have evaluated how well they would support an alerting use case.OBJECTIVE:The aim of this study was to evaluate the fitness of five existing standard anatomic terminologies to support similar or proximate alerts of an HIE-based prior CT alerting system.METHODS:We compared five standard anatomic terminologies (Foundational Model of Anatomy, Systematized Nomenclature of Medicine Clinical Terms, RadLex, LOINC, and LOINC/Radiological Society of North America [RSNA] Radiology Playbook) to an anatomic framework created specifically for our use case (Simple ANatomic Ontology for Proximity or Similarity [SANOPS]), to determine whether the existing terminologies could support our use case without modification. On the basis of an assessment of optimal terminology features for our purpose, we developed an ordinal anatomic terminology utility classification. We mapped samples of 100 random and the 100 most frequent LOINC CT codes to anatomic regions in each terminology, assigned utility classes for each mapping, and statistically compared each terminology's utility class rankings. We also constructed seven hypothetical alerting scenarios to illustrate the terminologies' differences.RESULTS:Both RadLex and the LOINC/RSNA Radiology Playbook anatomic terminologies ranked significantly better (P<.001) than the other standard terminologies for the 100 most frequent CTs, but no terminology ranked significantly better than any other for 100 random CTs. Hypothetical scenarios illustrated instances where no standard terminology would support appropriate proximate or similar alerts, without modification.CONCLUSIONS:LOINC/RSNA Radiology Playbook and RadLex's anatomic terminologies appear well suited to support proximate or similar alerts for commonly ordered CTs, but for less commonly ordered tests, modification of the existing terminologies with concepts and relations from SANOPS would likely be required. Our findings suggest SANOPS may serve as a framework for enhancing anatomic terminologies in support of other similar use cases.

BACKGROUND:Older adults are at risk for inadequate emergency department (ED) pain care. Unrelieved acute pain is associated with poor outcomes. Clinical decision support systems (CDSS) hold promise to improve patient care, but CDSS quality varies widely, particularly when usability evaluation is not employed.OBJECTIVE:To conduct an iterative usability and redesign process of a novel geriatric abdominal pain care CDSS. We hypothesized this process would result in the creation of more usable and favorable pain care interventions.METHODS:Thirteen emergency physicians familiar with the Electronic Health Record (EHR) in use at the study site were recruited. Over a 10-week period, 17 1-hour usability test sessions were conducted across 3 rounds of testing. Participants were given 3 patient scenarios and provided simulated clinical care using the EHR, while interacting with the CDSS interventions. Quantitative System Usability Scores (SUS), favorability scores and qualitative narrative feedback were collected for each session. Using a multi-step review process by an interdisciplinary team, positive and negative usability issues in effectiveness, efficiency, and satisfaction were considered, prioritized and incorporated in the iterative redesign process of the CDSS. Video analysis was used to determine the appropriateness of the CDS appearances during simulated clinical care.RESULTS:Over the 3 rounds of usability evaluations and subsequent redesign processes, mean SUS progressively improved from 74.8 to 81.2 to 88.9; mean favorability scores improved from 3.23 to 4.29 (1 worst, 5 best). Video analysis revealed that, in the course of the iterative redesign processes, rates of physicians' acknowledgment of CDS interventions increased, however most rates of desired actions by physicians (such as more frequent pain score updates) decreased.CONCLUSION:The iterative usability redesign process was instrumental in improving the usability of the CDSS; if implemented in practice, it could improve geriatric pain care. The usability evaluation process led to improved acknowledgement and favorability. Incorporating usability testing when designing CDSS interventions for studies may be effective to enhance clinician use.

Evidence suggests that geriatric patients may receive inadequate pain care in the emergency department (ED). Clinical decision support (CDS) systems are a potential solution and may improve the quality of pain care older ED patients receive with the use of targeted acute pain care interventions. We hypothesized that a novel electronic CDS system integrated into the ED electronic health record (EHR) may be effective in improving pain care for older adults with abdominal pain. This was a prospective, randomized controlled, comparative effectiveness study. A multi-component CDS intervention tool consisting of two electronic pain care alerts, an analgesic order set, a unique clinician documentation template, and reminders at patient discharge were integrated into a hospital ED EHR. The tool was randomized to resident emergency physicians over a 17-month period. Patient inclusion criteria were adults 65+ who presented to the ED with severe pain (10 on a pain scale of 0-10) and chief complaint of abdominal pain. Primary safety and pain care outcomes were analgesic use, total morphine equivalent dosages, times to first analgesic, and reduction in final pain scores during the ED visit. Multivariable logistic and linear regression analyses were completed comparing intervention to control subjects. Covariates include emergency severity index (ESI), sex, and race/ethnicity, and age. During the study period, a total of 2,388 adults 65+ years in age presented to the ED with abdominal pain; of these patients, the mean age was 75 (SD ± 8), 63% were female, 36% were Hispanic/Latino, 33% were white, and 21% were black. The mean ESI was 2.87 (SD ± 0.38). A total of 673 subjects (28%) met inclusion criteria (initial pain score 10); of these subjects, 179 were randomized to intervention and received at least one component of the CDS intervention. When comparing intervention versus control subjects (n=494), there was no increased safety risk of analgesic medication use (80% versus 79%, P=.67). Overall, there were no statistically significant differences in opioid analgesic use (66% versus 60%, P=.17), total morphine equivalent doses (7.7 versus 7.4mg, P=.14), time to first analgesic (153 versus 143 minutes, P=.45), or mean final pain scores (3.8 versus 4.4, P=.20). When comparing the intervention versus control groups by individual CDS components, however, subjects for whom the clinical documentation templates were used (n=61) were more likely to receive an opioid (OR 2.5, 95% CI 1.31, 4.05; 78% versus 60%, P=.004), and subjects who received either of the two pain care alerts had lower mean final recorded pain scores (3.57 versus 4.45, P=.03). These results remained significant in adjusted analyses. The general use of CDS interventions in an ED EHR to improve acute abdominal pain care for older adults appears safe. Components of the CDS such as clinical documentation templates and pain care alerts may be effective in promoting types of analgesic use and reducing overall final pain scores.

This paper focuses on improving the usability of an electronic health record (EHR) embedded clinical decision support system (CDSS) targeted to treat pain in elderly adults. CDSS have the potential to impact provider behavior. Optimizing CDSS-provider interaction and usability may enhance CDSS use. Five CDSS interventions were developed and deployed in test scenarios within a simulated EHR that mirrored typical Emergency Department (ED) workflow. Provider feedback was analyzed using a mixed methodology approach. The CDSS interventions were iteratively designed across three rounds of testing based upon this analysis. Iterative CDSS design led to improved provider usability and favorability scores.

XX Table of Contents–Part III SPARK: Personalized Parkinson Disease Interventions through Synergy between a Smartphone and a Smartwatch........................... 103 Vinod Sharma, Kunal Mankodiya, Fernando De La Torre, Ada Zhang, Neal Ryan, Thanh GN Ton, Rajeev Gandhi, and Samay Jain How Do Patient Information Leaflets Aid Medicine Usage? A Proposal for Assessing Usability of Medicine Inserts.......................... 115 Carla Galvao Spinillo Usability Improvement of a Clinical Decision Support System.......... 125 Frederick Thum, Min Soon Kim, Nicholas Genes, Laura Rivera, Rosemary Beato, Jared Soriano, Joseph Kannry, Kevin Baumlin, and Ula Hwang Information about Medicines for Patients in Europe: To Impede or to Empower?........... … Table of Contents–Part III XXI SMART Note: Student-Centered Multimedia Active Reading Tools for Tablet Textbooks................................................ 217 Jennifer George-Palilonis and …

Clinical decision support systems (CDSS) have the potential to impact provider behavior. Optimizing CDSS-provider interaction may enhance CDSS use. Five CDSS pain management interventions were developed and deployed in test scenarios within a simulated EHR that mirrored typical Emergency Department (ED) workflow. Provider feedback was analyzed using a mixed methodology approach. The CDSS interventions were iteratively designed across three rounds of testing based upon this analysis. Iterative design led to improved provider usability and favorability scores. Background: As the United States population ages, a greater proportion of ED visits will encompass geriatric patients. Examination of ED care of the geriatric population has revealed challenges in addressing pain symptoms. Unrelieved acute pain among geriatric patients is associated with poor outcomes such as increased morbidity and greater length of hospital stay. CDSS offer a promising approach to improve provider behavior in managing elder oligoanalgesia. Evaluation of CDSS within the normal provider workflow is often overlooked in the development of informatics interventions. This evaluation may help refine CDSS to reduce alert fatigue and frustration, thereby making CDSS a more valuable tool in guiding clinical care. This study sought to refine and optimize a geriatric abdominal pain care CDSS by using an iterative design process utilizing a test EHR environment. Methods: A group of five CDSS interventions was developed to address geriatric acute abdominal pain care throughout an ED visit (Table 1). Thirteen emergency physicians, all experienced in using the EHR, were recruited for participation. Over a 10-week period, seventeen 1-hour usability test sessions were conducted across 3 rounds of testing [Round 1: 5 users; Round 2: 5 users (1 repeat user from Round 1); Round 3: 7 users (2 repeat users from Round 2)]. Physicians were given 3 patient scenarios and were asked to provide simulated clinical care using the EHR, while interacting with the CDSS interventions. Users utilized order entry, documentation and discharge workflows. A System Usability Scale (SUS) survey (100 being perfect score), structured favorability questionnaire [scored negative (1), neutral (3), positive (5)], and open-ended narrative feedback of each CDSS intervention were completed after each user session. An interdisciplinary team reviewed structured and unstructured feedback and favorability scores after each round of testing, identified positive and negative issues in effectiveness, efficiency, and satisfaction, and then incorporated changes to the CDSS design. The SUS score was calculated for each round to quantify the overall effectiveness, efficiency and satisfaction for the CDSS. CDSS Intervention Description Type of Intervention Pain Score of 10 Alert A visual pop-up graphic in the center of the EHR screen that interrupts workflow and alerts provider that patient has pain score of 10 that has not been addressed Interruptive Alert Revaluate at 4 Hours Alert A visual stimulus within the order entry screen of the EHR that alerts provider that patient has pain score of 10 that has not been re-evaluated or addressed after 4 hours Non-Interruptive Alert Order Set A hyperlink that offers the provider a set of predefined analgesic treatment options for geriatric patients Non-Interruptive, interactive Medication Decision Aide HPI Reminder A statement that appears in history of present illness documentation reminding the provider to address patient’s pain Non-Interruptive, Non-Interactive Reminder or ‘Nudge’ Alert At Discharge A grayed out print button that prevents printing of discharge instructions for patients with an unaddressed pain score of 10 Interruptive Hard-Stop Alert Table 1. Description of Elderly Acute Abdominal Pain Care CDSS Interventions Figure 2. Example of CDSS iterative design over 3 rounds. There has been streamlining of text, clearer buttons and direct links to order sets or pain re-evaluation dialogs. (Four additional CDSS interventions not pictured due to space constraints on abstract) Results: After each round of CDSS evaluation, favorability scores and unstructured feedback responses were used to prioritize and target areas to improve usability, effectiveness, and the efficacy of the five CDSS interventions. See Figure 1 for a representative example of the changes that occurred (e.g., reduced text, fewer required fields, and direct links to actionable items) with the intervention for managing patient-reported pain scores of 10. Over the three testing rounds and redesigns, mean SUS scores improved as did mean favorability scores [scale 1 (worsened care) to 5 (improved care)] (Table 1). Round SUS Mean SUS Min SUS Max Fav. Mean Fav. Min Fav. Max n 1 75 64 87 3.4 2.2 4.2 5 2 81 70 96 2.9 1.8 3.8 5 3 89 69 100 4.5 4.4 4.7 7 Table 1. Mean System Usability Scores (SUS) and Favorability scores (Fav.) with minimum and maximum scores across three usability rounds showing iterative improvement. Limitations: Human-computer interactions are complex and may be considered in the larger socio-technical model. Although in this study EHR workflow was designed to mimic ED workflow, this study was conducted in an artificial environment that may not account for various other factors such as monitor alarms, patients and telephones competing for clinician’s attention. Though usability testing does not typically comprise a large number of trials, this study would be strengthened by a larger number of users completing the testing, none of who had participated in prior iterative rounds of testing. Though this study was conducted in one widely utilized commercial enterprise EHR, the interventions and effect may not transfer to other EHRs Conclusion: This study demonstrates how an iterative design process based on mixed methodology and implemented by an interdisciplinary team comprised of clinicians, research associates, and hospital information technology improves the usability and favorability of a geriatric pain care CDSS among providers. This process may be utilized in other institutions to improve provider satisfaction and enhance the use of CDSS. Clinical leadership has approved this CDSS implementation in the Mount Sinai Geriatric ED. Future studies will include real-world testing of the interventions to determine provider efficiency and clinical impact. Acknowledgements: This study was supported by: NIA grant # 5R21AG040734-02, Mount Sinai GCO# 10-1414(0001)(02)EM and New York State Empire Clinical Research Investigator Program