Objective To evaluate validity of hospital-discharged abstract administrative health data (DAD) over years through chart reviews. Methods We analyzed three chart review cohorts (4,008 patients in 2003, 3,045 in 2015, and 9,024 in 2022) in Calgary, Canada. Nurse reviewers determined the presence or absence of 17 clinical conditions employing similar protocols. The reviews were linked with DAD using a unique lifetime identifier, chart number, and admission date. We evaluated the validity of DAD, coded in ICD-10-Canada version, in recording conditions by comparing against chart reviews. The C-statistics was calculated in predicting in-hospital mortality. Results The mean difference in prevalence between chart reviews and DAD for these 17 conditions was 2.1% in 2003, 7.6% in 2015, and 6.3% in 2022. However, some conditions were relatively stable, such as diabetes (1.9%, 2.1%, and 1.1%) and metastatic cancer (0.3%, 1.1%, and 0.4%). For 17 conditions, the sensitivity ranged 39.6-85.1% in 2003, 1.3-85.2% in 2015, and 3.0-89.7% in 2022. The C-statistics for in-hospital mortality based on DAD was 0.84 in 2003, 0.81 in 2015, and 0.78 in 2022. Conclusion DAD increasingly under-coded conditions over 19 years. The validity of DAD decreased but remained relatively stable for certain conditions mandated for coding. The under-coding exerted minimal impact on in-hospital mortality prediction. Implications The under-coding could be primarily due to the increase of hospital patient volumes and the limited time allocated to coders. Consequently, there is a need to develop artificial intelligence methods based on electronic medical records to support coding practices and improve coding quality.

The International Classification of Diseases, Ninth Revision (ICD-9) was released in the 1970s and adopted in Canada for physician billing claims in 1979 (CIHI n.d.b.; WHO & International Conference for the Ninth Revision of the International Classification of Diseases 1977). ICD-9 is no longer adequate for representing our modern healthcare environment and patient needs. We summarize the findings from a small survey of ICD-9 users across Canada - such as family physicians, researchers and decision makers - who describe the limitations of ICD-9 and the features that they would desire in a new or updated classification system.

Introduction Countries use varying coding standards, which impact international coded data comparability. The `main condition' (MC) field is coded within the Discharge Abstract Database as "reason for admission" or "largest resource use". Objective We offer a preliminary analysis on the frequency of and contributing factors to MC definition agreements within an inpatient Canadian dataset. Methods Six professional coders performed a chart review between August 2016 and June 2017 on 3,000 randomly selected inpatient charts from three acute care hospitals in Calgary, Alberta. Coders classified the MC as "reason for admission", "largest resource use" or "both". Patients were admitted between 1st January and 30th June 2015 and met the inclusion criteria if they were >18 years, had an Alberta personal health care number, and had an inpatient visit for any service outside of obstetrics. Agreement between the two MC definitions was stratified by length of stay (LOS), emergency department admission, hospital of origin, discharge location, age, sex, procedures, and comorbidities. Chi-square analysis and frequency of inconsistencies were reported. Results Only 34 (1.51%) of the 2,250 patient charts had disagreeing MC definitions. Age, emergency visit on admit, LOS, hospital, and discharge location were associated with MC agreement. Chronic conditions were seen more often in MC definition agreements, and acute conditions seen within those disagreeing. Conclusion There was a small proportion of cases in which the condition bringing the patient to hospital was not also the condition occupying the largest resources. Within disagreements, further research using a larger sample size is needed to explore the presence of MC in a secondary/tertiary condition, the association between patient complexity and disagreeing MC definitions, and the nature of the conditions seen in the inconsistent MC definitions.

Introduction and Objectives: Electronic medical records (EMRs), specifically the discharge summary (DS), can improve secondary use data availability and interprofessional communication. We aimed to assess the completeness of our EMRs by assessing the presence of a DS in the EMR. Additionally, we assessed for indicators of a missing DS. Methods: A chart review was conducted on 3,011 inpatient charts in Calgary, Alberta. 893 charts were missing an electronic DS. A 10% sample was drawn to assess for presence of a paper DS. A Chi-square test, Fisher’s test and logistic regression assessed for associations between electronic DS absence and i) patient and hospital characteristics, and ii) patient comorbidities Results: The univariate analyses showed that age, being a surgical patient, a Charlson Comorbidity Index (CCI) of 1, as well as patients with myocardial infarctions, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, diabetes, and renal disease were associated with a missing DS. Those that were middle aged, surgical patients, or had fewer comorbidities were more likely to have a missing DS. Within the 10% sample, approximately 50% of all patients were from a surgical department, all of which were missing both electronic and paper discharge summaries. Conclusions: Our study is the first to describe indicators of missing electronic DS. The DS impacts interprofessional communication, patient outcomes, and data quality. Therefore, the implications of an incomplete DS are widespread. Our findings will caution future researchers using EMR data about the potential for incomplete data, particularly for patients who are surgical, middle aged, and have fewer comorbidities.

We present our preliminary results of an international survey on the practical adoption and use of the International Classification of Diseases (ICD) from a visualization and visual analytics perspective. The ICD system, in different versions, is globally used for coding morbidity and mortality statistics, however, coding practices vary across countries. Our survey includes questions about hospital data collection systems, use of features in ICD, and training of ICD coding specialists. Variations in ICD could hinder comparability and limit generalizability of observed findings. Our preliminary results establish the current state of ICD use and training internationally, and will ultimately be valuable to the World Health Organization to further research on how to improve ICD coding, and enhance international comparisons of health data. From a visualization and visual analytics perspective, the current differences in adoption and use of ICD poses challenges and opportunities. For example, when morbidity-data from two countries differ in their coding, can we still compare data from these countries, and if so, then under which circumstances? We discuss how visualization and visual analytics might help in these situations.

Background: The International Classification of Diseases (ICD) is globally used for coding morbidity and mortality statistics, however, its use, as well as the data collection features vary greatly across countries. Objective: To characterize hospital ICD-coded data collection worldwide. Methods: After an in-depth grey and academic literature review, an online survey was created to poll the 194 World Health Organization (WHO) member countries. Questions focused on hospital data collection systems and ICD-coded data features. The survey was distributed, using different methods, to potential participants that met the specific criteria, as well as organizations specialized in the topic, such as WHO Collaborating Centers (WHO-CC) or International Federation of Health Information Management Association (IFHIMA), to be forwarded to their representatives. Answers were analyzed using descriptive statistics. Results: Data from 48 respondents from 26 different countries has been collected. Results reveal worldwide use of ICD, with variations in the maximum allowable coding fields for diagnoses and interventions. For instance, in some countries there is an unlimited number of coding fields (Netherlands, Thailand and Iran), as opposed to others with only 1-6 available (Guatemala or Mauritius). Disparities also exist in the definition of a main condition, as 60% of the countries use “reason for admission” and 40% utilize “resource use”. Additionally, the mandatory type of data fields in the hospital morbidity database (e.g. patient demographics, admission type, discharge disposition, diagnoses, …) differ among countries, with diagnosis timing and physician information being the least frequently required. Conclusion: These survey data will establish the current state of ICD use internationally, which will ultimately be valuable to the WHO for the promotion of ICD and the rollout of ICD-11. Additionally, it will improve international comparisons of health data, and encourage further research on how to improve ICD coding.

IntroductionA high performing health data classification system requires clear, comprehensive code descriptions and user-friendly coding tools for effective coding. Coding specialists have essential specialized knowledge to contribute to the development and functionality of the 11th version of International Classification of Diseases (ICD-11) that will be released in June of 2018. Objectives and ApproachThe objective was to evaluate coding specialists’ experience of coding using ICD-11 for complete inpatient hospital charts. Mixed methods were employed for a survey and interviews. As part of a large field trial, 6 certified coding specialists underwent training to use the ICD-11 Beta Draft browser and ICD-11 Coding Tool. The coding team completed multiple coding exercises and coded over 60 charts each prior to evaluation of their experience. An electronic survey was used to evaluate ICD-11 knowledge, comprehension, and application of the coding training. Interviews explored the coders’ experience of learning and using the ICD-11 classification system. ResultsThe coding team (3 to 10 years of experience) received 14 hours classroom training and 5-10 hours per week of coding practice over 3 months. After training, perceived confidence in coding with ICD-11 was satisfactory; moderate (n=4), high (n=1), and low (n=1). Coding short scenarios was the most useful resource (n=6) and lack of guidelines was the most frustrating. Learning ICD-11 was deemed moderately (n=2) to somewhat (n=3) difficult but each coder described satisfaction in learning the new system. From the interviews, coders expressed liking the ability to more fully describe health conditions and hospital harms with code clusters. “The codes paint a clearer picture of what happened than with ICD-10”. With practice they achieved speed with the coding tools. Conclusion/ImplicationsCoding specialists learned and proficiently used the Beta Version of ICD-11 coding system with moderate perceived confidence. New ICD-11 codes and clustering functions allowed for more complete description of health scenarios and enhanced coder satisfaction.

IntroductionMain condition coding definitions in the International Classification of Disease (ICD) 10th and 11th versions are broadly defined in the current and upcoming versions of ICD, and coding health data can involve subjective coding specialist interpretation. Inconsistent coding can lead to inaccurate reporting, and lower quality data for research use. Objectives and ApproachMain condition coding agreement was compared between ICD-10 and ICD-11. 730 hospital charts were randomly selected from Foothills Medical Centre in Calgary, Alberta. These charts were previously coded using ICD-10, and six professional coding specialists recoded them using ICD-11. To compare frequencies of ICD-10 to ICD-11, we used current WHO crosswalk tables to match codes. For any missing codes, manual comparison by done by a qualified reviewer. In Canada, the “main condition” is the clinically significant reason for the hospital visit. If multiple problems were present, the diagnosis using the greatest amount of resources is coded, “main resource use”. ResultsOverall, 730 ICD-10 coded charts were analyzed. Of these charts, 79% (577) had matching resource coding between ICD-10 and ICD-11, and 21% (153) had mismatching coding. Matching coding was either considered an exact match between definitions (23.2%, 134), or similar but not identical (often one code has greater detail, 76.8%, 443). Mismatching codes were either due to different codes for similar conditions (13.1%, 20), different codes for not similar but related conditions (43.8%, 67), or completely different codes for unrelated conditions (43.1%, 66). Conclusion/ImplicationsICD-10 and ICD-11 main resource codes had a high match frequency indicating consistency between coding practices and ICD definitions (577/730, 79%). Future research will aim to understand underlying causes of mismatched main resource use codes. This research will help us understand issues in coding and contribute to future ICD-11 revisions.

IntroductionHospital safety performance is difficult to monitor when under-coding of hospital harms is common. The beta version of ICD-11 includes a 3-part model for coding harms to enhance adverse event descriptions. This method includes code clusters to detail each condition/event (e.g. bleed), cause (e.g. anticoagulant drug), and mode (over-dose). Objectives and ApproachThe study objective was to compare the proportion of adverse events captured using different classification systems. A large field trial of inpatient charts, previously coded in ICD-10 were coded with ICD-11. Coding training for the new ICD-11 focused on new codes, code clustering, and extension codes for cause and mode of the harm. Sensitivity, Specificity, NPV and PPV were reported for ICD-10 compared to ICD-11. ResultsOf the 1,009 records reviewed and coded using ICD-11 to date, 128 cases were coded as a harm in ICD-10 using our previously published PSI work. Coders identified 88 cases with the new ICD-11. Sensitivity and specificity were as follows: 31.3% and 94.6%. ICD-11 had NPV and PPV of 45.5% and 90.5% respectively compared to ICD-10. Detailed clinical comparison of mismatched codes will be completed. Study case examples will demonstrate advanced features of ICD-11, the coding rules being collaboratively developed by our team, CIHI, and WHO representatives, and potential analytic challenges. Conclusion/ImplicationsThe new ICD-11 found 8% of hospital admission were associated with a harm. Although the sensitivity was modest, specificity is quite high and correctly Identifies those cases without a harm. Clinical review of mismatched codes will provide further detailed code comparisons.

IntroductionIt is essential that clinical documentation and data coding be of high quality for the production of healthcare data for research or administrative purposes. However, there is a limited understanding of the facilitators and barriers of coded data quality and strategies to improve it. Objectives and ApproachOur objective was to qualitatively assess what influences coded data quality from the perspective of health information managers who are responsible for the work of coding specialists. Nine health information managers and/or coding quality coordinators who oversee coding specialists were identified and recruited from nine provinces across Canada to participate in this study. Semi-structured interviews were conducted which asked questions on participant demographics, responsibilities, data quality, costs and budget of coding, continuing education for Health Information Management (HIM), suggestions for quality improvement, and barriers to quality improvement. Interviews were recorded and transcribed, and analyzed using Directed Content Analysis methodology. ResultsInterviewees were primarily responsible for managing staff, quality assurance, audits, reporting, budget, data collection, and transcription. Managers reported that the experienced coders under their employ strengthened coding quality. Common barriers to coding quality included incomplete and unorganized chart documentation, which led to undercoding, and lack of communication and access to physicians for clarification when needed. Further, coding quality suffered as a result of limited resources (e.g. staffing and budget) being available to HIM departments for an ever-expanding workload, that was commonly due to increasingly complex charts and additional project data. Managers unanimously reported that coding quality improvements can be made by 1) making interactive training programs available to coding specialists, and 2) streamlining sources of information from charts (i.e., transitioning to standardized electronic charting). Conclusion/ImplicationsAlthough coding quality is generally regarded as high across Canada, quality can be hampered by incomplete and inconsistent chart documentation, lack of resources (e.g. financial support, staff, education), and inconsistent coding standards across hospitals and provinces. This study presents novel evidence for coding quality improvement across Canada.

IntroductionIn June 2018, the World Health Organization (WHO) will release the 11th Version of International Classification of Diseases (ICD-11). New training methods and materials are required. As a WHO Collaborating Center, with Canadian Institute for Health Information (CIHI) members, we trained 6 coding professionals for testing ICD-11 coding processes. Objectives and ApproachThe objective was to achieve a high level of inter-rater reliability using ICD-11 for acute care chart coding. We used Adult Learning principles with CIHI members and 6 certified coding specialists to co-create presentations, practice materials, and decision trees to teach knowledge and skill with ICD-11 tooling and content. Training involved 14 hours of interactive learning plus additional practice hours. A bank of questions and coding scenarios tested knowledge and application of ICD-11 terminology and principles. Coding was undertaken on a set of 3000 randomly selected inpatient Calgary hospital discharges as part of a large CIHR funded ICD-11 field trial. ResultsThe coding team achieved an average score of 84% on the ICD-11 coding quiz and 0.65 (0.33 -1.0) agreement on parent code of main condition for the coding quiz scenarios. 60 inpatient charts were coded by more than one coder to test inter-rater reliability. Agreement was ≧ 0.80 for the majority of parent codes for main condition. Coding differences may be due to unfamiliar code choices or training gaps. New code descriptions in ICD-11 enhance code selection. Challenges included training while codes were being built in the ICD-11 browser, and minimal coding rules or standards. Conclusion/ImplicationsRecommendations include more code descriptions in the browser and rules in a reference guide, teaching from simple to complex conditions, and multiple scenarios with ‘gold standard’ codes for practice. Reference Guide, Coding Tool, and Browser recommendations have been shared with members of the WHO Morbidity and Quality & Safety Advisory groups.

Theme: Data and Linkage QualityObjectives: To define health data quality from clinical, data science, and health system perspectives To describe some of the international best practices related to quality and how they are being applied to Canada’s administrative health data. To compare methods for health data quality assessment and improvement in Canada (automated logical checks, chart quality indicators, reabstraction studies, coding manager perspectives) To highlight how data linkage can be used to provide new insights into the quality of original data sources To highlight current international initiatives for improving coded data quality including results from current ICD-11 field trials Dr. Keith Denny: Director of Clinical Data Standards and Quality, Canadian Insititute for Health Information (CIHI), Adjunct Research Professor, Carleton University, Ottawa, ON. He provides leadership for CIHI’s information quality initiatives and for the development and application of clinical classifications and terminology standards. Maureen Kelly: Manager of Information Quality at CIHI, Ottawa, ON. She leads CIHI’s corporate quality program that is focused on enhancing the quality of CIHI’s data sources and information products and to fostering CIHI’s quality culture. Dr. Cathy Eastwood: Scientific Manager, Associate Director of Alberta SPOR Methods & Development Platform, Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB. She has expertise in clinical data collection, evaluation of local and systemic data quality issues, disease classification coding with ICD-10 and ICD-11. Dr. Hude Quan: Professor, Community Health Sciences, Cumming School of Medicine, University of Calgary, Director Alberta SPOR Methods Platform; Co-Chair of Hypertension Canada, Co-Chair of Person to Population Health Collaborative of the Libin Cardiovascular Institute in Calgary, AB. He has expertise in assessing, validating, and linking administrative data sources for conducting data science research including artificial intelligence methods for evaluating and improving data quality. Intended Outcomes:“What is quality health data?” The panel of experts will address this common question by discussing how to define high quality health data, and measures being taken to ensure that they are available in Canada. Optimizing the quality of clinical-administrative data, and their use-value, first requires an understanding of the processes used to create the data. Subsequently, we can address the limitations in data collection and use these data for diverse applications. Current advances in digital data collection are providing more solutions to improve health data quality at lower cost. This panel will describe a number of quality assessment and improvement initiatives aimed at ensuring that health data are fit for a range of secondary uses including data linkage. It will also discuss how the need for the linkage and integration of data sources can influence the views of the data source’s fitness for use. CIHI content will include: Methods for optimizing the value of clinical-administrative data CIHI Information Quality Framework Reabstraction studies (e.g. physician documentation/coders’ experiences) Linkage analytics for data quality University of Calgary content will include: Defining/measuring health data quality Automated methods for quality assessment and improvement ICD-11 features and coding practices Electronic health record initiatives

Introduction The beta version of the new ICD-11 includes a 3-part model for coding hospital acquired conditions (harms) to enhance adverse event descriptions. This method includes code clusters for detail each condition/event (e.g. bleed), cause (e.g. anticoagulant drug), and mode (over-dose). Objectives and Approach To compare the proportion of adverse events captured in ICD-11 to clinical chart review. A large field trial of 3000 inpatient charts are being coded with ICD-11 and chart review. Hospital admissions were randomly selected between January- June 2015 for adults at 3 Calgary hospitals. Chart reviewers were nurses trained to identify 11 categories of harms. Six coding specialists were trained to code with the ICD-11 3-part model for harm description. Coding decision trees and case examples of hospital-related harms were reviewed extensively by both teams. Coding training focused on new codes, code clustering, and extension codes for cause and mode of the harm. Results Of the 1,009 records reviewed and coded using ICD-11 to date, chart reviewers and coding specialists accurately identified 49 (37%) of the same charts with documented hospital harms. Both correctly identified 797 (91\%) of cases with no harm. Detailed analysis will follow. Study case examples will demonstrate advanced features of ICD-11 and the coding rules being collaboratively developed by our team, CIHI, and and WHO representatives. Conclusion/Implications Identification of hospital-related harms was consistent between coding specialists using ICD-11 principles and clinical chart reviewers. Variation existed in determining the cause and the mode of the harm. Case examples exemplify the new 3-part model for ICD-11 description of hospital-related harms.

IntroductionCoded data serves a critical part in the process of identifying the resource allocation required for each department in a hospital and for research purposes. This paper attempts a cost-benefit analysis of the transition from ICD-9 health indicator coding system to ICD-10 coding system and quantify the economic impacts. Objectives and ApproachThe hypothesis adopted by this paper is that the transition from ICD-9 to ICD-10 has been beneficial for the health system due better disease management, resulting in cost savings and facilitation of high quality health research. Analyzing the inflation-adjusted costs compared with the benefits accrued from implementing the new coding system would enable informed decision making for the stakeholders at government and other levels of health provision. The methodology involves constructing ‘benefit scenarios’ via analysis of existing literature and interviewing coding managers; costs are evaluated using data collected on re-training coders and productivity losses during the transition phase. ResultsAn example of a benefit scenario would take the form of cost savings associated with correctly identifying people with diabetes (due to coded charts), hence resulting in a decline in blood sugar (HbA1c) levels via better disease management. This in turn may cause reductions in other high blood-sugar related diseases and thus increase efficiency for government funding in the health care sector. Improved data quality in ICD-10 is expected to have resulted in gains from specificity due to increased sensitivity of data classification and grouping. Actual cost of re-training of coders and ICD-10 software provider fees are expected to be higher than the costs anticipated before ICD-10 implementation. Productivity losses in the transition phase are expected to have declined as coders became more adept at coding. Conclusion/ImplicationsAn economic evaluation proves to be a vital part of eliciting whether the transition to the newer method of coding, ICD-10, has been beneficial to the end users of the data. It is important to understand the efficiency of resource allocation to healthcare and the financial implications such investments entail.

Introduction Exposure to health care events sometimes has unintended and undesired consequences. Health care and complications arising in the course of care are diverse and complex. Representing them comprehensively in information systems is challenging, and presently beyond the bounds of practicality for routine administrative information systems that include ICD coded data. Objectives and Approach The ICD-11 conceptual model for hospital-acquired conditions has 3 components: 1) harm to patient 2) cause or source of harm and 3) mode or mechanism. A key feature of the Quality and Safety (Q\&S) code-set in ICD-11 is that a cluster of codes is required to represent an event or injury. Use of the term ‘cluster’ is novel in ICD-11 and so is the extent and the requirement for post-coordination. The cluster required to code a Q\&S case has three codes, one for each of the three components of the model given above. Results The first component, ‘harm’, is represented by an ICD–11 diagnosis code, from any chapter of the classification. Q\&S causes or sources of harm fall into 4 types that capture events caused by substances (drugs and medicaments, etc.), procedures, devices, and a mix of other types of causes (e.g. problems associated with transfusions, incorrect diagnosis, etc.). Q\&S ‘mode or mechanism’ refers to the main way in which the ‘cause’ leads to the ‘harm’ and are specific to the type of ‘cause’ (Table 1). Table 1 - Examples of corresponding Q\&S Mode or Mechanism Cause or Source of Harm Mode or Mechanism Substance Overdose, under-dose, wrong substance. Procedure Accidental perforation of an organ during a procedure. Device Dislodgement. Malfunction. Other cause Mismatched blood. Patient dropped during transfer from OR table. Conclusion/Implications This new conceptual model for coding healthcare-related harm, dependent on the clustering of codes, has great potential to improve the clinical detail of adverse event descriptions, and the overall quality of coded health data, for better monitoring and strategies for prevention.

IntroductionThe International Classification of Diseases (ICD) is globally used for coding morbidity statistics, however, its use, as well as the training provided to individuals assigning codes, varies greatly across countries. Objectives and ApproachThe goal is to understand the quality of coder training worldwide. After an in-depth grey and academic literature review, an online survey was created to poll the 194 World Health Organization (WHO) member countries. Questions focused on hospital data collection systems and the training provided to the coding professionals. The survey was distributed to potential participants that meet the specific criteria, as well as to organizations specialized in the topic, such as WHO-CC (WHO Collaborating Centers) and IFHIMA (International Federation of Health Information Management Association), to be forwarded to their representatives. Answers will be analyzed using descriptive statistics. ResultsThis ongoing project aims to capture responses from as many countries as possible, and thus far, data from 45 respondents from 20 different countries has been collected. Initial results reveal worldwide use of ICD, with variations in the maximum allowable coding fields for diagnoses and interventions. Coding specialists are the main personnel assigning codes, followed by physicians, and although minimum training is not mandatory in all countries (Sweden, Italy, Germany and Thailand), in those where it is, college/university degree is the most common requirement. Coding certificates most frequently entail passing a certification exam. Continuing education for coders is offered in all countries except one (Nigeria). Once more information is available, countries will be ranked and those depicting a better performance will be highlighted. Conclusion/ImplicationsThese survey data will establish the current state of ICD use and coding training internationally, which will ultimately be valuable to the WHO for the promotion of ICD and the rollout of ICD-11, for better international comparisons of health data, and for further research on how to improve ICD coding.