Aims: A multidisciplinary group of experts and patients developed the Model for ASsessing the value of Artificial Intelligence (MAS-AI) to ensure an evidence-based and patient-centered approach to introducing artificial intelligence technologies in healthcare. In this article, we share our experiences with meaningfully involving a patient in co-creating a research project concerning complex and technically advanced topics.Methods: The co-creation was evaluated by means of initial reflections from the research team before the project started, in a continuous logbook, and through semi-structured interviews with patients and two researchers before and after the active co-creation phase of the project.Results: There were initial doubts about the feasibility of including patients in this type of project. Co-creation ensured relevance to patients, a holistic research approach and the debate of ethical considerations. Due to one patient dropping out, it is important to foresee and support the experienced challenges of time and energy spent by the patient in future projects. Having a multidisciplinary team helped the collaboration. A mutual reflective evaluation provided insights into the process which we would otherwise have missed.Conclusions: We found it possible to create complex and data-intense research projects with patients. Including patients benefitted the project and gave researchers new perspectives on their own research. Mutual reflection throughout the project is key to maximise learning for all parties involved.

Learning activities are at the front-line of first impressions. In this paper, the education and training program for a large electronic health record transition project is presented. Management, and staff were interviewed before, during, and after implementation on their perception, reception, and benefit of various learning activities. Daily clinical work and obligations complicate adherence to learning programs, and the clinical professions differ in their approach to mandatory activities. Local learning activities empower staff, and planners should consider embedding room for adjustment of learning program during implementation.

Denmark is a high-income, progressive country known for its environmental stewardship and digital governance innovations. Less well known is its robust health information exchange (HIE) infrastructure. Rooted in an experiment to connect electronic health record systems in the early 1990s, today the nation boasts a robust, widely adopted platform for digital health. Laboratory, medication, and encounter data are centrally stored and leveraged to facilitate care in hospitals, physician practices, and nursing care facilities. Behind the scenes is an unimposing organization, MedCom, that has facilitated interoperability in Denmark for almost 30 years. This health information organization establishes national standards for data exchange, maintains the core infrastructure that facilitates exchange of clinical messages, and negotiates consensus-based HIE approaches to address health system challenges. In this case study, we examine the critical role MedCom plays in connecting national health IT platforms while describing the widely adopted components of the Danish national HIE infrastructure. We further describe the keys to success for HIE in Denmark, and we describe exciting, innovative HIE projects for the future in this Scandinavian nation.

OBJECTIVES:Artificial intelligence (AI) is seen as a major disrupting force in the future healthcare system. However, the assessment of the value of AI technologies is still unclear. Therefore, a multidisciplinary group of experts and patients developed a Model for ASsessing the value of AI (MAS-AI) in medical imaging. Medical imaging is chosen due to the maturity of AI in this area, ensuring a robust evidence-based model. METHODS:MAS-AI was developed in three phases. First, a literature review of existing guides, evaluations, and assessments of the value of AI in the field of medical imaging. Next, we interviewed leading researchers in AI in Denmark. The third phase consisted of two workshops where decision makers, patient organizations, and researchers discussed crucial topics for evaluating AI. The multidisciplinary team revised the model between workshops according to comments. RESULTS:The MAS-AI guideline consists of two steps covering nine domains and five process factors supporting the assessment. Step 1 contains a description of patients, how the AI model was developed, and initial ethical and legal considerations. In step 2, a multidisciplinary assessment of outcomes of the AI application is done for the five remaining domains: safety, clinical aspects, economics, organizational aspects, and patient aspects. CONCLUSIONS:We have developed an health technology assessment-based framework to support the introduction of AI technologies into healthcare in medical imaging. It is essential to ensure informed and valid decisions regarding the adoption of AI with a structured process and tool. MAS-AI can help support decision making and provide greater transparency for all parties.

Objective The successful development and implementation of sustainable healthcare technologies require an understanding of the clinical setting and its potential challenges from a user perspective. Previous studies have uncovered a gap between what emergency departments deliver and the needs and preferences of patients and family members. This study investigated whether a user-driven approach and participatory design could provide a technical solution to bridge the identified gap. Methods We conducted four workshops, and five one-to-one workshops with patients, family members, healthcare professionals, and information technology specialists to codesign a prototype. Revisions of the prototype were made until an acceptable solution was agreed upon and tested by the participants. The data were analyzed following iterative processes (plan → act → observe → reflect). Results The participants emphasized the importance of a person-centered approach focusing on improved information. An already implemented system for clinicians’ use only was redesigned into a unique patient module that provides a process line displaying continually updated informative features, including (1) person-centered activities, (2) general information videos, (3) a notepad, (4) estimated waiting time, and (5) the nurse and physician responsible for care and treatment. Conclusion Participatory design is a usable approach to designing an information system for use in the emergency department. The process yielded insight into the complexity of translating ideas into technologies that can actually be implemented in clinical practice, and the user perspectives revealed the key to identifying these complex aspects. The iterations with the participants enabled us to redesign an existing technology.

Citation for pulished version (APA): Helmark, L., Ahm, R., Andersen, C. M., Skovbakke, S. J., Kok, R. N., Wiil, U. K., Schmidt, T., Hjelmborg, J. V. B., Frostholm, L., Frydendal, D. H., Hansen, T. B., Zwisler, A. D. O., & Pedersen, S. S. (2021). Internet-based treatment of anxiety and depression in patients with ischaemic heart disease attending cardiac rehabilitation: a feasibility study (eMindYourHeart). European Heart Journal Digital Health, 2(2), 323–335. https://doi.org/10.1093/ehjdh/ztab037

Background Prediction of length of stay (LOS) at admission time can provide physicians and nurses insight into the illness severity of patients and aid them in avoiding adverse events and clinical deterioration. It also assists hospitals with more effectively managing their resources and manpower. Methods In this field of research, there are some important challenges, such as missing values and LOS data skewness. Moreover, various studies use a binary classification which puts a wide range of patients with different conditions into one category. To address these shortcomings, first multivariate imputation techniques are applied to fill incomplete records, then two proper resampling techniques, namely Borderline-SMOTE and SMOGN, are applied to address data skewness in the classification and regression domains, respectively. Finally, machine learning (ML) techniques including neural networks, extreme gradient boosting, random forest, support vector machine, and decision tree are implemented for both approaches to predict LOS of patients admitted to the Emergency Department of Odense University Hospital between June 2018 and April 2019. The ML models are developed based on data obtained from patients at admission time, including pulse rate, arterial blood oxygen saturation, respiratory rate, systolic blood pressure, triage category, arrival ICD-10 codes, age, and gender. Results The performance of predictive models before and after addressing missing values and data skewness is evaluated using four evaluation metrics namely receiver operating characteristic, area under the curve (AUC), R-squared score (R 2 ), and normalized root mean square error (NRMSE). Results show that the performance of predictive models is improved on average by 15.75% for AUC, 32.19% for R 2 score, and 11.32% for NRMSE after addressing the mentioned challenges. Moreover, our results indicate that there is a relationship between the missing values rate, data skewness, and illness severity of patients, so it is clinically essential to take incomplete records of patients into account and apply proper solutions for interpolation of missing values. Conclusion We propose a new method comprised of three stages: missing values imputation, data skewness handling, and building predictive models based on classification and regression approaches. Our results indicated that addressing these challenges in a proper way enhanced the performance of models significantly, which led to a more valid prediction of LOS.

Background One in five patients with ischaemic heart disease (IHD) develop comorbid depression or anxiety. Depression is associated with risk of non-adherence to cardiac rehabilitation (CR) and dropout, inadequate risk factor management, poor quality of life (QoL), increased healthcare costs and premature death. In 2020, IHD and depression are expected to be among the top contributors to the disease-burden worldwide. Hence, it is paramount to treat both the underlying somatic disease as well as depression and anxiety. eMindYourHeart will evaluate the efficacy and cost-effectiveness of a therapist-assisted eHealth intervention targeting depression and anxiety in patients with IHD, which may help fill this gap in clinical care. Methods eMindYourHeart is a multi-center, two-armed, unblinded randomised controlled trial that will compare a therapist-assisted eHealth intervention to treatment as usual in 188 CR patients with IHD and comorbid depression or anxiety. The primary outcome of the trial is symptoms of depression, measured with the Hospital Anxiety and Depression Scale (HADS) at 3 months. Secondary outcomes evaluated at 3, 6, and 12 months include symptoms of depression and anxiety (HADS), perceived stress, health complaints, QoL (HeartQoL), trial dropout (number of patients dropped out in either arm at 3 months) and cost-effectiveness. Discussion To our knowledge, this is the first trial to evaluate both the efficacy and cost-effectiveness of a therapist-assisted eHealth intervention in patients with IHD and comorbid psychological distress as part of CR. Integrating screening for and treatment of depression and anxiety into standard CR may decrease dropout and facilitate better risk factor management, as it is presented as “one package” to patients, and they can access the eMindYourHeart program in their own time and at their own convenience. The trial holds a strong potential for improving the quality of care for an increasing population of patients with IHD and comorbid depression, anxiety or both, with likely benefits to patients, families, and society at large due to potential reductions in direct and indirect costs, if proven successful. Trial registration The trial was prospectively registered on https://clinicaltrials.gov/ct2/show/NCT04172974 on November 21, 2019 with registration number [NCT04172974].

The number of deaths caused by alcohol-related diseases may be reduced by predicting alcohol use disorder (AUD). Many researchers have worked on AUD prediction using machine learning (ML) techniques. However, to the best of our knowledge, there is a lack of a comprehensive systematic literature review (SLR) that summarizes the existing studies on AUD prediction using ML in the last ten years. To address this knowledge gap, this article provides an SLR of academic articles on AUD prediction using ML techniques dated from January 2010 to July 2021. This SLR highlights technical decision analysis related to five aspects: data collection site, characteristics, and type of dataset; data sampling and data pre-processing techniques; feature types and feature engineering techniques; and characteristics of ML techniques and evaluation metrics. Six bibliographic databases were searched, and the identified studies were rigorously reviewed based on the above five aspects. In the selected studies, public datasets were not used very often for AUD prediction. Given that, the current paper identified two different types of data collection sites for review. Imbalanced class distribution in datasets was the primary focus of the pre-processing and sampling steps. Various features, including demographics, family history, drinking behaviour, and electronic health records, were introduced as the more widely used AUD prediction features. The filter, wrapper, and embedded methods were identified as the primary feature selection methods. Support vector machine was the most widely employed algorithm for predicting AUD; however, the lack of deep neural network techniques is evident in this field. Moreover, considering gender disparities, early detection of AUD, and identifying trajectories towards AUD are suggested for future work. For the purpose of evaluating the performance of the prediction approaches, most studies considered the overall accuracy and the area under the receiver operating characteristic curve. Nevertheless, external validation was not performed in any of the selected studies. This paper also discusses challenges and open issues of AUD prediction for future research. This SLR represents a valuable resource for scholars investigating the prediction of AUD.

Information Security Awareness among employees in healthcare has become an essential part in safeguarding health information systems against cyber-attacks and data breaches. We present three simple security awareness questions that can be included in larger surveys gauging other aspects of information systems. The questions have been tested in a national Danish survey to evaluate correlations among medical profession, computer proficiency, experience, and place of employment. We find that dissatisfaction with system usability is strongly linked with reduced information security awareness, and that clinical professions have different responses to security concerns.

Abstract Aims Anxiety and depression are prevalent in 20% of patients with ischaemic heart disease (IHD); however, treatment of psychological conditions is not commonly integrated in cardiac rehabilitation (CR). Internet-based psychological treatment holds the potential to bridge this gap. To examine the feasibility of an eHealth intervention targeting anxiety and depression in patients with IHD attending CR. Methods and results We used a mixed-methods design, including quantitative methods to examine drop-out and change in anxiety and depression scores, and qualitative methods (thematic analysis) to evaluate patients’ and nurses’ experiences with the intervention. The therapist-guided intervention consisted of 12 modules provided via a web-based platform. The primary outcome was drop-out, with a drop-out rate <25% considered acceptable. Patients were considered as non-drop-out if they completed ≥5 modules. Out of 60 patients screened positive for anxiety and/or depression, 29 patients were included. The drop-out rate was 24% (7/29). Patients had a mean improvement in anxiety and depression scores of 5.5 and 4.6, respectively. On average, patients had 8.0 phone calls with their therapist and 19.7 written messages. The qualitative analysis of patients’ experiences identified four themes: treatment platform, intervention, communication with therapist, and personal experience. Patients were positive towards the intervention, although some found the assignments burdensome. From the nurses, we identified three themes: intervention, inclusion procedure, and collaboration with study team. The nurses were positive, however, due to limited time some struggled with the inclusion procedure. Conclusion Integrating an eHealth intervention in CR is feasible and the drop-out rate acceptable.

ObjectivesThis systematic review aimed to assess the performance and clinical feasibility of machine learning (ML) algorithms in prediction of in-hospital mortality for medical patients using vital signs at emergency departments (EDs).DesignA systematic review was performed.SettingThe databases including Medline (PubMed), Scopus and Embase (Ovid) were searched between 2010 and 2021, to extract published articles in English, describing ML-based models utilising vital sign variables to predict in-hospital mortality for patients admitted at EDs. Critical appraisal and data extraction for systematic reviews of prediction modelling studies checklist was used for study planning and data extraction. The risk of bias for included papers was assessed using the prediction risk of bias assessment tool.ParticipantsAdmitted patients to the ED.Main outcome measureIn-hospital mortality.ResultsFifteen articles were included in the final review. We found that eight models including logistic regression, decision tree, K-nearest neighbours, support vector machine, gradient boosting, random forest, artificial neural networks and deep neural networks have been applied in this domain. Most studies failed to report essential main analysis steps such as data preprocessing and handling missing values. Fourteen included studies had a high risk of bias in the statistical analysis part, which could lead to poor performance in practice. Although the main aim of all studies was developing a predictive model for mortality, nine articles did not provide a time horizon for the prediction.ConclusionThis review provided an updated overview of the state-of-the-art and revealed research gaps; based on these, we provide eight recommendations for future studies to make the use of ML more feasible in practice. By following these recommendations, we expect to see more robust ML models applied in the future to help clinicians identify patient deterioration earlier.

To design and evaluate a mental health treatment program and internet-based delivery platform for patients with ischemic heart disease (IHD) attending cardiac rehabilitation with the aim of reducing the risks associated with anxiety and/or depression. Patients diagnosed with IHD and comorbid anxiety and/or depression. Participatory design of treatment program and internet platform through staged inclusion of participants in two groups. Group 1 was enrolled as co-researchers with prolonged engagement in the project. Group 2 participated only in the pilot evaluation workshop. Three patients were included in Group 1, two patients in Group 2. Inclusion of patients proved challenging, but the extended collaboration with co-researchers yielded valuable circumstantial insight and resulted in the design of a novel nine-module treatment program. Additionally, the inclusion of two participant groups helped shape the development of an internet platform based on an open-source content management system. Our grouped participation method contributes with several recommendations and reflections of advantages of this approach. Collaboration with co-researchers helped us gain a deeper understanding of the impact of language on self-perception and potential stigma. Prolonged participation led to a higher level of trust and familiarity, which enabled uncovering of issues otherwise hidden.

Length of Stay (LOS) prediction at the time of admission can give clinicians insight into the illness severity of patients and enable them to prevent complications and adverse events. It can also help hospitals to manage their facilities and manpower more efficiently. This paper first applies Borderline-SMOTE and multivariate Gaussian process imputer techniques to overcome data skewness and handle missing values which have been ignored by most studies. Then, based on our conversation with clinicians, patients are stratified into five classes according to their LOS. Finally, five machine learning algorithms, including support vector machine, deep neural networks, random forest, extreme gradient boosting, and decision tree are developed to predict LOS of unselected patients admitted to the emergency department at Odense University Hospital. These models utilize information of patients at the time of admission, including age, gender, heart rate, respiratory rate, oxygen saturation, and systolic blood pressure. Performance of predictive models on the data before and after imputation and class balancing are investigated using the area under the curve metric and the results show that our proposed solutions for data skewness and missing values challenges improve the performance of predictive models by an average of 13%.

Many people take prescription medications and need information about the risks and benefits associated with taking them. Citizens are increasingly turning to the internet for health information and medication information is no exception. There are a variety of websites that offer Online Medication Information for Citizens (OMIC). This study compared six such websites using the Health Literacy Online (HLO) Checklist as a framework. Additionally, we conducted a detailed analysis of the individual content in each OMIC for three different medications. We identified several strengths and weaknesses of the different websites in terms of how they were designed and written and their appropriateness for users with limited eHealth literacy.

The information system landscape in Emergency Departments is almost as diverse as the patients treated there. Clinicians and management rely heavily on having access to timely and accurate information about arriving and admitted patients. Implementing and evaluating novel research-based systems in such settings will inevitably face several challenges. Any information technology implementation project faces the generic challenge of designing a system that addresses the actual problem in a manner acceptable for the end-users. Additionally, several contextual challenges emerge that are specific to the targeted domain. In this paper, we present our approach to the design and implementation of a novel system and the experiences gained along the way. We provide recommendations for challenges relating to external and internal issues and operational perspectives, namely: 1) Coupling streams of data from disconnected systems, 2) Coping with insufficient vendor APIs, 3) Sensible interaction with external systems, 4) Dealing with the intermittent nature of medical device utilization, 5) Capturing data from disconnected devices, 6) Enabling washout handling to support study design, and 7) Monitoring the deployed system during operation.

Downtime of information systems is a universal challenge faced by health care institutions. Regardless of whether downtime is planned or unplanned, the unavailability of essential information requires alternative solutions. In this paper, we conduct a scoping review of how hospitals deal with downtime. A total of 13 papers were included in the final analysis, and we found that coping can be grouped into three strategies; 1) Increasing communication, 2) Analog fallback, and 3) Restricted redundant systems. As the majority of coping mechanisms are related to increasing communication and analog fall back, our findings point to the importance of customizing coping mechanisms for individual healthcare institutions.