Context Many patients at community-based health centers experience chronic conditions and face barriers in accessing technologies that can help with the management of these conditions. Remote patient monitoring (RPM) is an effective telehealth method for supporting chronic condition management, but little prior research has assessed how community clinics can implement RPM among their patients. Objective Using lessons learned from evaluation of an RPM program in three communitybased health centers, we describe the EHR tools and implementation strategies subsequently developed to better support future RPM programs in the same clinic network. Study Design and Analysis Mixed methods explanatory sequential evaluation. Qualitative data included semi-structured interviews, analyzed thematically using an implementation framework. Quantitative data were extracted from the clinics' shared EHR and analyzed descriptively. Setting Three community-based health care organizations from the multi-state OCHIN network. Population Studied Health center staff and patients at clinics participating in the RPM program. Intervention/Instrument RPM program tools and implementation strategies. Outcome Measures RPM implementation and health impact. Results The three participating health centers implemented the RPM programs in different ways, and only one distributed sufficient devices for outcome measure assessment. Though associated health outcomes were promising when RPM devises were fully utilized, the RPM program faced barriers to adoption including a lack of EHR tools to support device orders and reporting, difficulties with device integration, and a lack of training and educational materials for diverse different patient populations. Evaluation findings were leveraged to address these barriers in future RPM programs by developing and refining related EHR tools and program implementation supports. Conclusions The experiences of three community-based health organizations indicate that while RPM technology holds promise for addressing their patients' chronic disease management needs, successful and equitable RPM implementation requires substantial investment in implementation support and the development of EHR tools to support the use of RPM. The new EHR tools and implementation support developed in the processes described here will be evaluated in a future prospective assessment.
Context: Electronic health record (EHR) systems are regularly updated and revised to provide new functionalities to support care delivery, yet frequent changes could lead to EHR users overlooking potentially helpful improvements. Prior research on best practices for EHR change communications and how they can be integrated into existing workflows is limited, yielding a knowledge gap on how different approaches impact the adoption of new or revised EHR tools. Objective: To evaluate user perceptions of revisions made to a change communication program of an EHR shared by a national network of community-based health care organizations. Study Design and Analysis: Qualitative evaluation involving semi-structured interviews. Transcripts were analyzed thematically, informed by the Integrated Technology Implementation Model framework. Setting: Community-based health care organizations from the multi-state OCHIN network. Population Studied: Health center staff and EHR change subject matter experts from OCHIN. Intervention/Instrument: EHR change communications program. Outcomes Measures: EHR change implementation and adoption. Results: Participants characterized the new change communication program as involving processes that enable communication to different users about changes made to the EHR. Health center staff were satisfied overall with the content and conduits by which EHR changes are communicated (e.g., being provided multiple opportunities to learn about the changes and their impacts, change management support, and their customization toward role type). Reported barriers were related to knowledge gaps of available tools to support implementation, and the frequency of the EHR changes themselves. Clinic staff participants reported feeling overwhelmed by the frequency of EHR modifications, which made it difficult to prepare for changes and their integration into clinic workflows. Conclusions: In this setting, existing EHR change communication approaches are effective, but tension remains between the need for continual EHR updates and barriers that health center staff face in frequently adopting and implementing such changes. Future work should focus on refining this timing and further assessment of the support required to implement necessary EHR changes in a manner that does not burden clinical teams.
Context: Building a source for pan-Canadian EMR data, which has a complex and geographically varied healthcare system, is challenging. For more than a decade, the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) has been working to develop and standardize primary care data to ensure it is sufficient quality to be a valuable source for clinicians, researchers, and policy makers. A data quality (DQ) framework was developed to evaluate the CPCSSN database. Objective: to assess two DQ dimensions (1) accuracy and reliability; and (2) comparability and coherence, using evidence-based indicators. Study Design and Analysis: Three indicators were used: (a) element presence-the completeness of common data elements expected to be present or 'not null'; (b) data source agreement-how information derived from CPCSSN compared to other sources of information; and (c) data across jurisdictions and sources- the prevalence of common data elements across sites, EMR type and province. We used data that included records up until June 30, 2022. Outcome measure: (a) % present of common data elements within the database; (b) prevalence of common chronic diseases; and (c) prevalence of common ICD-9 codes, medication codes and lab codes. Results: Coded fields within CPCSSN are ≥93% complete for demographic elements. Diagnostic data is highly present in uncoded fields (<6% null) but shows some missingness in coded fields (~75% present). Medication and lab names are well captured (> 99% present) but medication specifications (ex. duration, frequency) need standardization. The prevalence of common chronic diseases estimated using CPCSSN data are reasonable and comparable to estimates from administrative and survey data. Comparing common diagnostic, medication, and lab codes across site, EMR type and province shows that there is a great degree of variation in the use of these common codes at each site, which is influenced by EMR type and province. Conclusions: The CPCSSN database has reasonable DQ in terms of accuracy and reliability, and comparability and coherence when it is used for epidemiological research. The indicators highlight the extensive work CPCSSN has done to create coded, standardized information. We recommend CPCSSN operations continues to develop cleaning and processing tools to reduce missingness in coded fields. It is recommended that users request identification of site, EMR and province so that clustering can be accounted for in the analysis.
Context: We developed MARVIN, an artificial intelligence-based chatbot to engage people with HIV in their primary care and support their HIV self-management. Objective: To assess its usability and identify the barriers and facilitators to its acceptance. Study Design and Analysis: A 4-week pilot study using mixed methods. Setting: McGill University Health Centre (Montreal, Canada). Population studied: People with HIV on regular treatment. Intervention/Instrument & Outcome Measures: Participants were asked to have at least 20 conversations within 3 weeks with MARVIN on predetermined topics and then, to complete the Usability Metric for User Experience-lite (UMUX-lite) and Acceptability E-Scale (AES) surveys. Observed mean scores were compared with predetermined thresholds (68/100 and 24/30, respectively). Qualitatively, randomly selected participants were invited to semi-structured focus groups/interviews to discuss their experiences with MARVIN. Verbatim transcriptions were deductively coded using the constructs of the Consolidated Framework for Implementation Research. Barriers and facilitators were identified according to the four subconstructs of the Technology Acceptance Model (TAM): perceived ease of use, perceived usefulness, attitude toward use, and behavioral intention to use. Results: From April to December 2021, 28 participants completed the questionnaires. Their mean age was 40.2 years (SD=11.7), most were male (n=24/28), and over half (n=15/28) preferred to communicate with MARVIN in English. Mean scores for the UMUX-lite and AES were 69.9 and 23.8, both were not significantly below their respective thresholds (p=.76 and p=.42). Nine participants were interviewed. Identified facilitators included user-friendliness, accessibility across devices, confidentiality with a sense of security, and reliability of the information provided. However, lack of topics and functions, limited comprehension, and lack of usage guidance and support were identified as barriers, along with its implementation on only a single platform, Facebook Messenger. Conclusions: MARVIN is easy to use, useful, and acceptable as a self-management tool for People with HIV. The qualitative results highlight the enhanced accessibility of relevant information and sense of interaction and safety using MARVIN as facilitating its usability and acceptance, while the quality of information provided, and the technology's adaptability are factors that require further attention.
Context eConsult is an online service through which primary care practitioners (PCP) submit patients' cases to specialists and receive a recommendation. eConsult reduces patients' unnecessary in-person visits and improves timely access to specialists' advice. Objective We explored the extent of reusability of past specialists' responses as answers to current and future PCP questions. Study Design and Analysis We used natural language processing and machine learning (ML) experimentations to cluster the most similar questions. We then automatically built an answer bank through summarization of the associated past specialists' answers to the topmost similar PCP questions. This machine generated answer was then presented to enable PCPs to proactively refine their question prior to submitting it to the specialist, if still needed. Setting or Dataset We randomly selected a sample of 3000 PCP questions and the associated answers from eConsult, submitted during 2020. Population Studied The population is an unbiased random sample of patient cases for whom PCP asked a question from the specialist. Intervention/Instrument The online software system is the means of intervention-instrumentation to provide care more efficiently and effectively rather than requiring in-person visits for all. Outcome Measures We used ROUGE to measure the accuracy of the correspondence among the terms in the generated response and those in the actual specialist response to evaluate the results. Results We aggregated past specialists' answers associated with the similar questions to automatically generate relevant answers and evaluated their quality against actuals. We summarized the associated answers to the topmost 5 similar questions and compared them with the actual answers. The top ROUGE measures were 19% and 12% for Neurology-Migraine and Pediatrics respectively. Conclusion The combination of clustering and summarization of the past conversations can play an important role in monitoring, maintenance and re-usability of medical knowledge. There is still an identifiable gap between the machine-generated medical answers and the actual answers using eConsult's historical data as the diversity and variability of the new questions are still higher than what the summary of aggregation of answers to those in the past can sufficiently provide and be specific enough to fully qualify to replace human answers.
Context This study aimed to address physician burnout related to EHR and teamwork by designing and implementing a dual-focused intervention: reducing EHR workload and enhancing care team cohesion. The intervention included quick action shortcuts in EHR and a 30-second compassion team practice (CTP) during daily physician-nurse dyad huddles. Objectives Design and implement a dual-focused intervention to reduce EHR workload and enhance care team cohesion by improving management of electronic communication with patients, other clinicians thereby strengthening care teams and mitigating physician burnout. Study Design and Analysis Modified stepped wedge clustered randomized trial. Dependent variables included dichotomized 1-item burnout level, a perceived ease of EHR work scale, Mini Z subscale for supportive workplace, and the number of inbasket messages. Population studied Forty-five physicians, including 16 family physicians, 11 general internists, and 18 subspecialists, in 12 clinics. Intervention Forty-five physicians were randomized by their clinic to intervention (first EHR-only, then EHR+CTP) versus standard work, followed by EHR-only, then EHR+CTP, over four 4-week periods. Intent-to-treat analyses used mixed effects models. Random effects were included to account for the clustering of clinic and physician. Dataset EHR and survey data. Outcome Measures The outcome measures of this study include the proportion of burnout, perceived ease of EHR work, the Mini Z subscale for supportive workplace, and the number of inbasket messages. Results The study found that while there was no significant impact on burnout, both the EHR intervention (coefficient=0.76, p=0.01) and EHR+CTP intervention (coefficient=0.80, p<0.01) were associated with higher perceived ease of EHR work, and the Mini Z supportive work environment subscale marginally significantly increased with EHR+CTP (coefficient=0.61, p=0.07), indicating an increase in perceived supportiveness in the workplace. Additionally, the total number of inbasket messages per week declined (coefficient=-48.3, p=0.03) after the EHR intervention. Conclusions Researchers and informatics leaders collaborated to catalyze changes in inbasket management and team relationship. While no significant impact on burnout was observed in adjusted analyses, encouraging findings were obtained on perception of ease of EHR work, a more supportive workplace, while number of messages declined.
Context This study explores the potential application of artificial intelligence (AI) in facilitating communication in electronic health record (EHR) systems to reduce the burden and risk of clinician burnout. We leveraged previously extracted real EHR patient messages from a study of physician burnout, generated responses using ChatGPT, and then qualitatively compared them to actual physician responses. Objectives Assess the potential use of AI in reducing clinician burnout caused by electronic messaging by generating responses to patient messages using ChatGPT. The study also evaluates the AI-generated responses based on their relational connection, informational content, recommendations for next steps, and the extent of editing required before they can be used. Study Design and Analysis Qualitative analysis of AI-generated responses to patient messages compared to actual physician responses. Dataset EHR messages Population studied EHR patient messages Intervention Previously extracted real EHR patient messages were used as prompts to generate responses using ChatGPT. Qualitative comparisons were made between the generated responses and actual physician responses for different categories of patient messages, evaluating their relational connection, informational content, follow up recommendations, and the amount of editing needed. Outcome Measures Outcome measures include the qualitative assessments of ChatGPT-generated responses to patient messages compared to actual physician responses. Results The study found that AI-generated responses lacked relational connection, appearing mechanical and impersonal, while physicians' responses varied widely, ranging from personal and empathic to instrumental and prescriptive. The informational content of AI-generated responses was also general, compared to physicians' responses, which were more specific. Additionally, AI-generated responses were on average three times longer than physicians' responses and required substantial editing. Recommendations from AI were generally generic, while physicians provided tailored recommendations based on the patient's specific needs. Conclusions While some users have started using generative AI language models in healthcare communication, this study demonstrates significant challenges to making them useful to clinicians, and more efforts are needed to harness the potential of AI to support human critical thinking, judgment, and creativity in healthcare.
Context: There is a global movement to advance the use of data to inform and improve health service delivery. Despite extensive and long term computerisation of Australian general practice, aggregated data has rarely been used to improve the quality of primary care. This project arose from an increasing interest from a large Academic Health Science Centre (AHSC) in data-led health care improvement in primary care. The AHSC, Monash Partners, commissioned this work to inform future data initiatives. Objective: Identify contextual influences, key challenges and approaches required to embed data-led primary health care improvement in the Monash Partners’ region. Study Design and Analysis: Our qualitative design used semi-structured interviews. We used purposive and snowball sampling to recruit 24 clinicians, researchers and policy makers between October and November 2021. Interviews were conducted via Zoom video-conferencing and recorded. Analysis was iterative, with thematic coding developed using deductive and inductive processes and refined during analysis, reflection and investigator discussions. Setting: The AHSC’s catchment, South East and East Melbourne, Australia, between October and November 2021. Population Studied: Regional data custodians and data users. Results: We found an uncoordinated system that mirrored Australia’s difficulties in using health data to benefit society. Nearly all participants were passionate about the potential for data-led health care improvement, yet cautious about the practicalities of change. General practitioner (GP) academics stressed how difficult it was for GPs to see a ‘value add’ from primary care data, and whether data initiatives met GP and community needs. Nevertheless, participants saw potential in creation of an accessible high quality data asset linking GP clinical data with state and federal health datasets. This resource would require a regional data strategy and formalised partnerships between regional primary care organisations, data custodians and academics. Conclusions: Our findings suggest that a region-wide strategy involving creative individual and organisational capacity building could achieve success in sustainable primary care data-led improvement. This should focus on the needs of diverse communities, and the priorities of primary care clinicians and their teams. The AHSC appears to be the only regional organisation with the mandate and capacity to promote such an approach.
Context: Canada is investing in initiatives to improve primary care. To measure their impact, performance measurement systems require a comprehensive set of health indicators. To date, most primary care health indicators measure process of care, disease, and health service utilization, with a gap in measures of health outcomes. Function is a measure of patient health that could measure outcomes. Regularly measuring function in primary care has had limited success. For primary care teams to implement and use measures of function, they need to be appropriate (i.e. timely, meaningful and credible) and to be feasible in primary care. Objectives: To identify the most appropriate and feasible measures of function, for adult patients, that can be used as health indicator(s) in primary care. Design: Classic Delphi Setting: Primary care in Canada Population Studied: Expert panel: 12 Canadian academic leaders, with expertise/experience in team-based care, primary care, patient function, and/or performance measurement. Intervention: Rounds 1-3 identified potential measures of function and sought consensus on a finite set of (4-5) measures. Round 4 measured levels of agreement on the appropriateness and feasibility of using 5 patient-reported health measures (SF-36, SF-12, EQ-5D-5L, WHODAS 2.0, and WHOQOL BREF) to measure function in primary care. Outcome Measures: Round 1-3: Percent of respondents that would keep, modify, or remove a proposed measure with consensus set at 75%. Round 4: The percent of respondents that rated, on a 5-point Likert scale, the appropriateness and utility of the 5 measures, and the percent of respondents who ranked the measures from 1 (best) to 5 (worst). Results: Round 1-3: 41 potential measures were identified representing the 3 ICF domains. Consensus was reached to remove 13 measures with no consensus achieved for the remaining 28. Round 4: Measures rated the highest for appropriateness were the SF-12 (80%) and the SF-36 (70%). Measures rated the highest for feasibility were the SF-12 (100%) and the EQ-5D-5L (90%). Measures with the highest overall rankings were the SF-12 (90%) and the EQ-5D-5L (60%). Conclusions: Measuring function is complex with all domains of function deemed important to measure. All 5 patient-reported health measures were deemed at least slightly appropriate and feasible. The SF- 12 was shown to be the most appropriate and feasible measure of function that could be used as a health indicator in primary care.
Context: Asthma is a prevalent chronic disease that is difficult to manage and associated with marked disparities in outcomes. Among the most visible disparities is the higher rate of visits to the Emergency Department (ED) for uncontrolled asthma involving the most at-risk patients. One promising approach to addressing disparities is Shared Decision Making (SDM), a process by which the patient and provider jointly make a healthcare choice. SDM is associated with improved outcomes for patients; however, time constraints and availability of staff are noted implementation barriers. The use of health IT solutions may increase the adoption of SDM. Coach McLungsSM is an interactive web-based application that engages pediatric patients and their caregivers in a personally tailored experience and collects patient-reported data. The background logic then incorporates the complex asthma guidelines to determine the level of severity or control and pulls forward tailored guideline-based treatment recommendations for both the patient and provider in two respective summaries, which provides decision support for both audiences in developing a shared decision around the treatment plan. Objective: The goal of this study is to evaluate the implementation of the Coach McLungsSM intervention into primary care. Study Design and Analysis: Stepped wedge randomized control study design with a baseline control period and 5 intervention steps over 3 years. Setting: 21 pediatric and family medicine practices within a large, integrated, nonprofit healthcare system based in Charlotte, NC. Population Studied: Patients between 7-17 years old with an asthma diagnosis. Intervention: Implementation will be guided using the Expert Recommendations for Implementing Change (ERIC), a compilation of implementation strategies, and evaluated using CFIR (the Consolidated Framework for Implementation Research) and RE-AIM (Reach Effectiveness, Adoption, Implementation, Maintenance). Outcome Measures: We will measure changes in emergency department visits, hospitalizations, and oral steroid use, which serve as surrogate measures for patient-centered asthma outcomes. Conclusions: We anticipate that the tailored implementation of Coach McLungsSM across primary care practices will lead to a decrease in emergency department visits, hospitalizations, and oral steroid use for patients in the intervention group as compared to the control arm.
ContextThe eConsult service is an online system that primary care practitioners (PCP) submit questions concerning their patients’ care to specialists and receive a response within one week. Responses can include suggestions for treatment, recommendations for referral, or requests for additional information.ObjectiveThe objective is to examine the use of natural language processing (NLP) in automatic identification of frailty in patient cases and explore semantic characteristics of such cases that distinguished them from non-frailty cases to help refine and inform the definition of frailty and provide its reusable knowledge.Study DesignWe conducted a machine learning (ML) experimentation using NLP to assess frailty in a dataset of cases submitted through eConsult Service. We used text data prepared and vetted by the experts, as training material to achieve accurate prediction of “frail” cases automatically.Setting or DatasetIn this study, we selected contrasting samples from eConsult cases submitted in the Champlain health region between 2018 and 2019 and filtered them by patient’s age 65 and the use of term frail or frailty in their case description. Non-frailty cases were selected among younger patients below 65 years old in whom PCP did not refer to as frail.Population StudiedPCPs can keep communicating with the specialist until they are satisfied with the answers to close the case. The population is an unbiased random sample of PCP communications, that were then labeled.Outcome MeasuresWe used 10-fold cross-validation to measure precision, recall, F1 and accuracy to evaluate the results.ResultsUsing text from electronic conversations between primary care providers and specialists we developed a champion algorithm (Random Forest with Bag-of-Words as data representation) that predicted frailty cases with an accuracy of 94% (+/− 0.12). This study also provided evidence of semantic characteristics that were specific to frail cases. Among several frail cases, the most frequent topic-terms were related to medications: daily dose, recommended medication and best treatment.ConclusionIt was possible to automatically identify Frailty cases in eConsult system with high accuracies using NLP. Noteworthy is that no other patient data were required. The predictions can inform and assist practitioners recognize frailty in patient cases and as a result, provide organized and reusable knowledge to enhance the quality of service for frail patients.
Context: Proxy online health information seekers informally seek online health information on behalf of or because of others without necessarily being asked to do so. Proxy information seekers in a person’s social circle may help this person overcome information-seeking barriers (e.g., low level of e-health literacy) and illness challenges (e.g., when they are too physically weak to search themselves). However, little is known of the context, use, and reported outcomes of proxy online health information seeking. Objective: To explore the outcomes of proxy online health information seeking behaviour. Study Design: Mixed studies literature review with framework synthesis, then a convergent mixed methods study. Setting: In partnership with Naître et Grandir (an online parenting resource). Population: Members of the entourage of parents of 0- to 8-year-old children (grandparents, family members, friends, neighbours, or professionals working with children). Instrument: Validated ‘Information Assessment Method’ (IAM) questionnaire. Results: An initial theoretical model was developed using findings from the mixed studies review. IAM questionnaire responses showed that entourage members who actively seek online information were more likely to report that the information can help them be less worried, handle a problem, and decide what to do with someone else. In the interviews, entourage members reported proxy-seeking for reassurance, out of personal curiosity, as part of their professional role, or following an explicit request from the parents. They used the information to provide informational support (either by sharing the webpage directly or discussing its content), or to provide material support for a child in their care (e.g., playing games with a child), or to provide emotional support. In some cases, they did not share the information to avoid tensions with the parents in question. Positive outcomes included improvement in the relationship with others, and negative outcomes included interpersonal tensions when information was unsolicited. The initial model was revised to incorporate findings from the mixed methods study. Expected Outcomes: Findings from this study advance knowledge on proxy online health information seeking behaviour and outcomes. Health care practitioners can target patients’ entourage with information for dissemination and use. Patients can be encouraged to turn to their entourage for support using online health information.
BACKGROUND:Health measurement guides policies and health care decisions are necessary to describe and attain the quintuple aim of improving patient experience, population health, care team well-being, health care costs, and equity. In the primary care setting, patient-reported outcome measurement allows outcome comparisons within and across settings and helps improve the clinical management of patients. However, these digital patient-reported outcome measures (PROMs) are still not adapted to the clinical context of primary health care, which is an indication of the complexity of integrating these tools in this context. We must then gather evidence of their impact on chronic disease management in primary health care and understand the characteristics of effective implementation. OBJECTIVE:We will conduct a systematic review to identify and assess the impact of electronic PROMs (ePROMs) implementation in primary health care for chronic disease management. Our specific objectives are to (1) determine the impact of ePROMs in primary health care for chronic disease management and (2) compare and contrast characteristics of effective ePROMs' implementation strategies. METHODS:We will conduct a systematic review of the literature in accordance with the guidelines of the Cochrane Methods Group and in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for its reporting. A specific search strategy was developed for relevant databases to identify studies. Two reviewers will independently apply the inclusion criteria using full texts and will extract the data. We will use a 2-phase sequential mixed methods synthesis design by conducting a qualitative synthesis first, and use its results to perform a quantitative synthesis. RESULTS:This study was initiated in June 2022 by assembling the research team and the knowledge transfer committee. The preliminary search strategy will be developed and completed in September 2022. The main search strategy, data collection, study selection, and application of inclusion criteria were completed between October and December 2022. CONCLUSIONS:Results from this review will help support implementation efforts to accelerate innovations and digital adoption for primary health care and will be relevant for improving clinical management of chronic diseases and health care services and policies. TRIAL REGISTRATION:PROSPERO International Prospective Register of Systematic Reviews CRD42022333513; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=333513. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID):DERR1-10.2196/48155.
ABSTRACT Introduction Learning health systems (LHS) use data to improve care. Descriptive epidemiology to reveal health states and needs of the LHS population is essential for informing LHS initiatives, including development of decision support tools. To properly characterize complex populations, both simple statistical and artificial intelligence techniques can be useful. We present the first large-scale description of the population served by one of the first primary care LHS in North America. Objectives Our objective is to describe sociodemographic, clinical, and health care use characteristics of adult primary care clients served by the Alliance for Healthier Communities, which provides team-based primary health care through Community Health Centres (CHCs) across Ontario, Canada. Methods Using electronic health record data from 2009-2019 for all CHCs, we perform table-based summaries for each characteristic; and apply unsupervised leaning techniques to explore patterns of common condition co-occurrence, care provider teams, and care frequency. Results Of the 221,047 eligible clients, those at CHCs that primarily serve those most at risk (homeless, mental health, addictions) tend to have more chronic conditions and social determinants of health, which are also prominent in clients with multimorbidity. Most care is provided by physician and nursing providers, with heterogeneous combinations of other provider types. A subset of clients have many issues addressed within single-visits and there is within- and between-client variability in care frequency. Example methodological considerations learned for future LHS initiatives include the need to carefully consider the level of analysis and associated implications for data quality and target population, heterogeneity in conditions and care characteristics, and non-uniform risk profiles across the care history. Conclusions We demonstrate the use of methods from statistics and artificial intelligence, applied with an epidemiological lens, to provide an overview of a complex primary care population. In addition to substantive findings, we discuss implications for future LHS initiatives.
Context: Goal-oriented models of care are becoming more widely used as part of primary care delivery for older adults with multimorbidity and complex care needs. While these models hold promise, implementation remains challenging. Digital health solutions may improve adoption however, they require evaluation to determine feasibility and impact. Objective: This study evaluates the implementation and effectiveness of the electronic Patient Reported Outcome (ePRO) mobile application and portal system, designed to enable goal-oriented care delivery in inter-professional primary care practices. Study design: Multi-method pragmatic randomized control trial using a stepped-wedge design and ethnographic case studies over a 15-month period. Setting: 6 comprehensive primary care practices across Ontario. Population studied: Older adults with complex care needs; target sample 176 patients. Intervention: Patient and provider participants used the ePRO tool in addition to usual care. The 6 practices randomized into either early (3-month control; 12-month intervention) or late (6-month control; 9-month intervention) groups. Outcome measures: The Assessment of Quality of Life-4D collected at baseline and 3-month intervals. Ethnographic data (observations and interviews) collected at mid-point and end of the intervention. Outcome data were analyzed using linear models. Ethnographic data was analyzed using qualitative description and framework analysis methods, guided by Normalization Process Theory. Results: The trial experienced recruitment challenges resulting in fewer sites (n=6) and participants (n=45) than expected. As such the impact of ePRO on quality of life could not be definitively assessed; analysis trends suggest decreased quality of life for patients over both the control and intervention periods. Ethnographic data reveals a complex implementation process, in which the meaningfulness (or coherence) of the technology to individuals lives, relationships and approach chronic disease management drove adoption and perceived value or irrelevance of ePRO. Conclusions: Implementation challenges were broad and largely unexpected. The difficultly in aligning meaningfulness of a complex intervention across diverse user groups over time, suggests the intervention may not be sufficiently adaptable, or that more dynamic trial methods may be required. Including ethnographic data collection reveals critical underlying mechanisms driving digital health innovations.
CONTEXT: Artificial intelligence (AI) is increasingly being recognized as having potential importance to primary care (PC). However, there is a gap in our understanding about where to focus efforts related to AI for PC settings, especially given the current COVID-19 pandemic. OBJECTIVE: To identify current priority areas for AI and PC in Ontario, Canada. STUDY DESIGN: Multi-stakeholder engagement event with facilitated small and large group discussions. A nominal group technique process was used to identify and rank challenges in PC that AI may be able to support. Mentimeter software was used to allow real-time, anonymous and independent ranking from all participants. A final list of priority areas for AI and PC, with key considerations, was derived based on ranked items and small group discussion notes. SETTING: Ontario, Canada. POPULATION STUDIED: Digital health and PC stakeholders. OUTCOME MEASURES: N/A. RESULTS: The event included 8 providers, 8 patient advisors, 4 decision makers, 3 digital health stakeholders, and 12 researchers. Nine priority areas for AI and PC were identified and ranked, which can be grouped into those intended to support physician (preventative care and risk profiling, clinical decision support, routine task support), patient (self-management of conditions, increased mental health care capacity and support), or system-level initiatives (administrative staff support, management and synthesis of information sources); and foundational areas that would support work on other priorities (improved communication between PC and AI stakeholders, data sharing and interoperability between providers). Small group discussions identified barriers and facilitators related to the priorities, including data availability, quality, and consent; legal and device certification issues; trust between people and technology; equity and the digital divide; patient centredness and user-centred design; and the need for funding to support collaborative research and pilot testing. Although identified areas do not explicitly mention COVID-19, participants were encouraged to think about what would be feasible and meaningful to accomplish within a few years, including considerations of the COVID-19 pandemic and recovery phases. CONCLUSIONS: A one-day multi-stakeholder event identified priority areas for AI and PC in Ontario. These priorities can serve as guideposts to focus near-term efforts on the planning, development, and evaluation of AI for PC.
Context: The effective deployment of artificial intelligence (AI) in primary health care requires a match between the AI tools that are being developed and the needs of primary health care practitioners and patients. Currently, the majority of AI development targeted toward potential application in primary care is being conducted without the involvement of these stakeholders. Objective: To identify key issues regarding the use of AI tools in primary health care by exploring the views of primary health care and digital health stakeholders. Study Design: A descriptive qualitative approach was taken in this study. Fourteen in-depth interviews were conducted with primary care and digital health stakeholders. Setting: Province of Ontario, Canada Population studied: Primary health care and digital health stakeholders Outcome Measures: N/A Results: Two main themes emerged from the data analysis: Worth the Risk as Long as You Do It Well; and, Mismatch Between Envisioned Uses and Current Reality. Participants noted that AI could have value if used for specific purposes, for example: supporting care for patients; reducing practitioner burden; analyzing existing evidence; managing patient populations; and, supporting operational efficiencies. Participants identified facilitators of AI being used for these purposes including: use of relevant case studies/success stories with realistic uses of AI highlighted; easy or low risk applications; and, end user involvement. However, barriers to the use of AI included: data quality; digital divide/equity; distrust of AI including security/privacy issues; for-profit motives; need for transparency about how AI works; and, fear about impact on practitioners regarding clinical judgement. Conclusion: AI will continue to become more prominent in primary health care. There is potential for positive impact, however there are many factors that need to be considered regarding the implementation of AI. The findings of this study can help to inform the development and deployment of AI tools in primary health care.
Context: Use of trustworthy online consumer health information (OCHI) is generally associated with benefits, yet barriers such as low health literacy may reduce these benefits. One of the largest groups of OCHI consumers is parents of young children. In addition to OCHI, parents reach out to their social circle for tailored advice, emotional support, and culturally relevant parenting information. However, little is known, about the use of parenting OCHI by the parents' social circle. Objective: To uncover OCHI outcomes when members of parents' social circle search for OCHI. Study Design: Convergent mixed methods study. Setting: Online parenting and child health information newsletter and magazine, available at naitreetgrandir.com (N&G). Population studied: Participants who completed a questionnaire about the information presented by N&G between April 13th, 2019 to March 30th, 2021. Instrument: The validated Information Assessment Method (IAM) questionnaire, implemented by N&G since 2015. Main outcome measures: IAM responses on OCHI outcomes by participants were analyzed using descriptive statistics, and responses compared between parents and their entourage (grandparents, family, friends and neighbours). Qualitative component: 14 parents' entourage members were interviewed to uncover perceived OCHI outcomes (up to saturation) and thematic analysis was conducted. Quantitative and qualitative components were conducted and analysed separately; results were compared using a joint display to provide a complete picture. Results: 51,320 completed IAM questionnaires (81% by parents) were included in the analysis, pertaining to 1079 N&G webpages (mean 47.6; range 1-637). Entourage members were more likely than parents to report using the information in a discussion with someone else and this led them to being better able to make a decision with those persons. Interviewees described sharing the information they found in some situations or not sharing it to avoid conflict. They used the information to provide support or do something for the parent(s) or child. Sharing information led to improved decision making, improved relationships, less worry and better health outcomes, or in some cases led to tensions. Conclusion: By better understanding how people use information together, health information can be adapted to meet both individual and group needs. Public health interventions aimed at supporting parents can do so by facilitating shared decision making.
In this paper, we propose the first deep reinforcement learning framework to estimate the optimal Dynamic Treatment Regimes from observational medical data. This framework is more flexible and adaptive for high dimensional action and state spaces than existing reinforcement learning methods to model real life complexity in heterogeneous disease progression and treatment choices, with the goal to provide doctor and patients the data-driven personalized decision recommendations. The proposed deep reinforcement learning framework contains a supervised learning step to predict the most possible expert actions; and a deep reinforcement learning step to estimate the long term value function of Dynamic Treatment Regimes. We motivated and implemented the proposed framework on a data set from the Center for International Bone Marrow Transplant Research (CIBMTR) registry database, focusing on the sequence of prevention and treatments for acute and chronic graft versus host disease. We showed results of the initial implementation that demonstrates promising accuracy in predicting human expert decisions and initial implementation for the reinforcement learning step.
