IntroductionDuring the COVID-19 pandemic, electronic health record (EHR) data has been used to investigate disease severity and risk factors for severe COVID-19 in people with multiple sclerosis (pwMS). Methodological challenges including sampling bias, and residual confounding should be considered when conducting EHR-based studies. We aimed to address these limitations related to the use of EHR data in order to identify risk factors, including the use of disease modifying therapies (DMTs), associated with hospitalization for COVID-19 amongst pwMS.MethodsWe performed a retrospective cohort study including a sample of 47,051 pwMS using a large US-based EHR and claims linked database. Follow-up started at the beginning of the pandemic, February 20th 2020, and continued until September 30th 2020. COVID-19 diagnosis was determined by the presence of ICD-10 diagnostic code for COVID-19, or a positive diagnostic laboratory test, or an ICD-10 diagnostic code for coronaviruses. We used Cox regression modeling to assess the impact of baseline demographics, MS disease history and pre-existing comorbidities on the risk of hospitalization for COVID-19. Then, we identified 5,169 pwMS using ocrelizumab (OCR) and 3,351 pwMS using dimethyl fumarate (DMF) at baseline, and evaluated the distribution of the identified COVID-19 risk factors between the two groups. Finally, we used Cox regression models, adjusted for the identified confounders, to estimate the risk of hospitalization for COVID-19 in pwMS treated with OCR compared to DMF.ResultsAmong the pwMS cohort, we identified 799 COVID-19 cases (1.7%) which resulted in 182 hospitalizations for COVID-19 (0.4%). Population differences between the pwMS and COVID-19 cohorts were observed. Statistical modeling identified older age, male gender, African-American race, walking with assistance, non-ambulatory status, severe relapse requiring hospitalization in year prior to baseline, and specific comorbidities to be associated with a higher risk of COVID-19 related-hospitalization. Comparing the COVID-19 risk factors between OCR users and DMF users, MS characteristics including ambulatory status and MS subtype were highly imbalanced, likely arising from key differences in the labelled indications for these therapies. Compared to DMF use, in unadjusted (HR 1.58, 95% CI 0.73 - 3.44), adjusted (HR 1.28, 95% CI 0.58 - 2.83), propensity score weighted (HR 1.25, 95% CI 0.56 - 2.80), and doubly robust models (HR 1.29, 95% CI 0.57 - 2.89), no significantly increased risk of hospitalization for COVID-19 was associated with OCR use.ConclusionWe observed significant population differences when comparing all pwMS to COVID-19 cases, as well as significant differences in key confounders between OCR and DMF treated patients. In unadjusted analyses we did not observe a statistically significant higher risk of COVID-19 hospitalization in pwMS treated with OCR compared to DMF, with further attenuation of risk when adjusting for the key confounders. This study re-emphasises the importance to appropriately consider both sampling and confounding bias in EHR-based MS research.

BACKGROUND:Huntington's disease (HD) is a rare neurodegenerative disease that presents with progressive psychological, cognitive and motor impairment. These diverse symptoms place a high burden on the patient, families and the healthcare systems they rely on. This study aimed to describe the epidemiology and clinical burden in individuals with HD compared with controls from the general population.METHODS:This cohort study utilised data from general practitioner medical records to estimate the prevalence and incidence of HD between January 2000 and December 2018. A cohort of incident HD cases were matched 1:3 to controls from the general population, in whom common clinical diagnoses, medications and healthcare interventions were compared at the time of first recorded diagnosis and at a time close to death. Incidence rates of common diagnoses and mortality were compared with matched controls in the time following HD diagnosis.RESULTS:Prevalence of HD increased between 2000 and 2018, whilst incidence remained stable. Prevalence of psychiatric diagnoses and symptomatic treatments were higher in HD cases than controls. A higher relative risk of psychotic disorders, depression, insomnia, dementia, weight loss, pneumonia and falls was observed in HD cases. Risk of death was >4 times higher in HD, with a median survival of ~12 years from first recorded diagnosis.CONCLUSIONS:This study demonstrates the significant and progressive clinical burden in individuals with HD up to 18 years after first recorded diagnosis.

Background: Hemophilia A (HA) is an X-linked condition characterized by factor VIII (FVIII) levels below the normal range, leading to excessive bleeding. Disease severity can be categorized as mild (FVIII: >5-40%), moderate (FVIII: 1-5%) or severe (FVIII: <1%) (Blanchette et al. J Thromb Haemost 2014). Females represent a minority of patients with HA and their bleeding phenotypes are poorly characterized in scientific literature. In this retrospective cohort study, the real-world, patient-centric PicnicHealth database, which integrates clinical data with patient-reported outcomes, is used to examine the epidemiology and bleeding phenotype in females with mild HA (FwHA) compared with a cohort of males with mild HA (MwHA). Methods: FwHA and MwHA have been recruited since June 2020 in the United States. Severity of HA was based on physician-reported provider notes, or baseline FVIII levels if provider notes were inconclusive (for this analysis, mild HA was defined as FVIII >5-50%). Demographics, disease characteristics, and patient-reported outcomes were collected. Bleeding phenotype was assessed up to a year prior to enrollment and described according to bleed etiology (i.e., traumatic, spontaneous, procedure-related, unknown), bleed frequency, and bleed management (use of FVIII concentrates). A two-tailed test was used to analyze differences between the cohorts, with p<0.05 considered significant. Results: At data cut-off on May 31, 2022, 68 patients with mild HA had been recruited: 23 FwHA and 45 MwHA (Table 1A). There were no differences in racial demographics, with approximately 70% identifying as 'White' in both groups. The median FVIII level was significantly higher in the female cohort at 26% (Q1, Q3: 17, 36%), compared with the male cohort at 10% (Q1, Q3: 6, 13%; p=<0.001). Median (Q1, Q3) age at first diagnosis was also significantly higher in the female cohort, at 24.3 (8.71, 36.0) years, compared with the male cohort, at 10.5 (1.02, 21.4) years (p=0.048). However, after adjusting for median FVIII levels, the difference in age at first diagnosis was not statistically significant (p=0.56) between males and females. Median (Q1, Q3) follow-up periods available from medical records were similar for FwHA and MwHA (8.92 [5.60, 12.9] years and 9.22 [4.27, 12.4] years, respectively). FVIII inhibitors were not identified in any FwHA but were present in 13.3% of MwHA. The majority of patients’ records had no evidence of HIV, hepatitis B, or hepatitis C status; however, 5.9% of MwHA were reported to be positive for hepatitis B and C, compared with no FwHA. Prevalence of iron deficiency anemia was higher in FwHA compared with MwHA, at 27.7% vs. 6.7% (significance not assessed due to low numbers). Access to hemophilia treatment centers was numerically higher for FwHA compared with MwHA (82.6% vs. 73.3%), but this difference was not significant. Evaluation of bleeding phenotype in the female and male cohorts is described in Table 1B. The small number of recorded events does not allow for meaningful conclusions about the differences in bleeding phenotypes between females and males with mild disease. The proportion of FwHA treated with FVIII prophylaxis (30.4%) was numerically lower than the MwHA (40.0%), but not statistically significant (p=0.611). For patients on prophylaxis in the year before enrollment, the median (Q1, Q3) number of days on prophylaxis was 209 (149, 268) days for FwHA and 365 (168, 365) days for MwHA (significance not assessed due to low numbers). On-demand FVIII concentrate usage was similar in both groups with 47.8% of FwHA and 51.1% of MwHA receiving FVIII. Conclusions: This real-world cohort study observed differences in disease characteristics between FwHA and MwHA, but statistical analysis was limited by low participant numbers. FwHA had higher baseline FVIII levels compared with MwHA. On-demand use of FVIII concentrates was comparable between the cohorts. Recorded bleeding events were too low in number to draw conclusions regarding bleeding phenotype. As this database relies on patient participation, there was a potential selection bias towards those with more symptomatic disease, and some data were incomplete. Future studies will analyze ongoing prospective data, collect more granular data on bleeding phenotype, and conduct comparative analyses of cohorts stratified by baseline FVIII levels. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal

Background: People with mild or moderate hemophilia A (HA) constitute 40-52% of all people with HA; nonetheless, they are a poorly characterized population in real-world data and published literature (Witkop et al. Blood 2021). To better understand this population, we used a novel platform, PicnicHealth, which integrates real-world data from patients' medical records and patient-reported outcomes, including bleeding occurrence, treatment for bleeds, and pain (Skinner et al. Blood 2021; Witkop et al. Blood 2021). We aim to study the epidemiology of mild and moderate HA and evaluate the effects of various demographic variables and the COVID-19 pandemic on use of healthcare facilities. Methods: People with HA (PwHA) have been enrolled into the PicnicHealth online record management platform since June 2020. Inclusion criteria for this cohort with mild or moderate HA were based on baseline factor (F)VIII activity level (mild, >5-50%; moderate, 1-5%). Records were collected from all known healthcare providers and facilities, including hematology, primary care, and other specialty care, in addition to laboratory reports, imaging reports, pathology, and any inpatient or outpatient encounter since the diagnosis. Descriptive statistical analyses were performed to summarize cohort characteristics. Results: As of June 2022, there were 113 PwHA enrolled in PicnicHealth who met the inclusion criteria for this analysis, of whom 68 (60.2%) had mild HA and 45 (39.8%) had moderate HA. In total, 90 (79.6%) participants were male and 23 (20.4%) were female. The majority identified as 'White' (n=77; 68.1%), followed by 'More than one race or unreported' (n=25; 22.1%), 'Black or African American' (n=8; 7.1%), and 'other race' (n=3; 2.7%). The median age at diagnosis was 12.5 years (Q1, Q3: 1.96, 29.8). Only 8 participants (7.1%) had evidence of FVIII inhibitors during their lifetime. The median (Q1, Q3) Hemophilia Joint Health Score was 4.00 (3.00, 9.00) and the median (Q1, Q3) annualized bleed rate was 0.17 (0, 3.63). In terms of treatment, in the 2 years prior to enrollment, 36 (31.9%) participants had received FVIII prophylaxis, 73 (64.6%) had received FVIII on demand, and 10 (8.8%) had received emicizumab (Table 1). In the 2 years prior to enrollment, participants with mild or moderate HA visited a healthcare facility a median (Q1, Q3) of 7 (4, 11) times and a hematology practitioner a median (Q1, Q3) of 2 (1, 3) times. The majority of the visits to hematology practitioners were to medical doctors (93%), with a smaller number to nurse practitioners or allied health professionals (7%). There was no notable difference in the median number of visits to healthcare facilities or hematologists according to disease severity, race, or inhibitor status. The five states with the highest number of healthcare facility visits by hemophilia patients were California (24.7%), Indiana (7.1%), Michigan (5.9%), Texas (5.9%), and Virginia (4.3%). Finally, we assessed the impact of the COVID-19 pandemic on participants' visits to healthcare facilities. Between January 2019 and July 2021, there were no apparent changes to emergency care visits. For routine care visits, there have been variations since the pandemic began, which may be attributable to lockdown restrictions, although the number of these does not strictly correlate with COVID-19 case numbers (Figure 1). The study is limited by potential bias resulting from participant reporting and the retrospective nature of data collection. Conclusions: Our study provides valuable insights into the characteristics and utilization of healthcare of the under-represented cohort of people with mild or moderate HA. We found notable variation in the geographical distribution of healthcare facilities and visits by PwHA across different states. As expected, routine visits by patients fluctuated during the COVID-19 pandemic. The data generated from this study may aid planning for, and delivery of, the most appropriate and effective care for this group of patients. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal

Background Where a sufficiently powered placebo-controlled clinical study is a challenge, it may be possible to use external natural history data as a comparator arm (‘external control’) to determine treatment efficacy.1 Aim To establish feasibility of using Enroll-HD as an external control by benchmarking it to the placebo arm of the GENERATION HD1 (NCT03761849) study. Methods/Techniques GENERATION HD1 inclusion criteria were applied to construct a comparable cohort from Enroll-HD. Nearest-neighbour propensity-score matching and inverse probability weighting (IPTW-ATT) methods were implemented to balance baseline characteristics between GENERATION HD1 and Enroll-HD cohorts. Regression models computed differences in annual decline of cUHDRS, TFC, TMS, SDMT, SWRT for both GENERATION HD1 placebo and Enroll-HD cohorts after controlling for CAP, CAG, age and baseline performance. Results/Outcome Both methods balanced baseline characteristics between participants in the Enroll-HD cohort and the GENERATION HD1 placebo (standardised mean difference <0.1). Superior covariate balance was achieved by IPTW-ATT (Figure 1). One-year clinical decline was greater in the Enroll-HD cohort compared with the GENERATION HD1 placebo arm (Figure 2). Conclusions Faster clinical decline in the Enroll-HD cohort compared with the GENERATION HD1 placebo arm has implications for the feasibility of using Enroll-HD as an external comparator in drug trials, since this could erroneously inflate the observed treatment effect. Slower decline in the placebo arm may be attributed to placebo effect and selection bias. Future work should explore the consistency of findings across placebo arms in Huntington’s disease trials and consider how best to utilise external data to enrich placebo cohorts. Reference Jahanshahi M, Gregg K, Davis G, Ndu A, Miller V, Vockley J, et al. The Use of External Controls in FDA Regulatory Decision Making. Therapeutic Innovation & Regulatory Science 2021;55:1019-35.

BACKGROUND Huntington's disease (HD) is a rare, genetic, neurodegenerative disease. Obtaining population-level data on epidemiology and disease management is challenging. OBJECTIVE To investigate the epidemiology, clinical manifestations, treatment, and healthcare utilization of patients with HD in Israel. METHODS Retrospective population-based cohort study, including 20 years of routinely collected data from Maccabi Healthcare Services, an insurer and healthcare provider for one-quarter of the Israeli population. RESULTS The study cohort included 109 adult patients (aged ≥18 years) diagnosed with HD, with mean age of 49.9 years and 56%females. The most common HD-related conditions were anxiety (40%), behavioral problems (34%), sleep disorders (21%), and falls (13%). Annual incidence rates for HD ranged from 0.17 to 1.34 per 100,000 from 2000 to 2018; the 2018 crude prevalence in adults was 4.36 per 100,000. Median survival from diagnosis was approximately 12 years (95%CI: 10.4-15.3). The most frequent symptomatic treatments were antidepressants (69%), antipsychotics (63%), and tetrabenazine (63%), the only drug approved for the treatment of HD chorea in Israel during the examined period. Patterns of healthcare utilization changed as disease duration increased, reflected by increased frequency of emergency department visits and home visits. CONCLUSION This retrospective population-based study provides insights into the prevalence, incidence, clinical profile, survival, and resource utilization of patients with HD in ethnically diverse Israel. The findings in this study are generally consistent with the international literature and demonstrate the value of routinely collected healthcare data as a complementary resource in HD research.

PURPOSE Adherence to tamoxifen citrate among women diagnosed with metastatic breast cancer can improve survival and minimize recurrence. This study aimed to use real-world data and machine learning (ML) methods to classify tamoxifen nonadherence. METHODS A cohort of women diagnosed with metastatic breast cancer from 2012 to 2017 were identified from IBM MarketScan Commercial Claims and Encounters and Medicare claims databases. Patients with < 80% proportion of days coverage in the year following treatment initiation were classified as nonadherent. Training and internal validation cohorts were randomly generated (4:1 ratio). Clinical procedures, comorbidity, treatment, and health care encounter features in the year before tamoxifen initiation were used to train logistic regression, boosted logistic regression, random forest, and feedforward neural network models and were internally validated on the basis of area under receiver operating characteristic curve. The most predictive ML approach was evaluated to assess feature importance. RESULTS A total of 3,022 patients were included with 40% classified as nonadherent. All models had moderate predictive accuracy. Logistic regression (area under receiver operating characteristic 0.64) was interpreted with 94% sensitivity (95% CI, 89 to 92) and 0.31 specificity (95% CI, 29 to 33). The model accurately classified adherence (negative predictive value 89%) but was nondiscriminate for nonadherence (positive predictive value 48%). Variable importance identified top predictive factors, including age ≥ 55 years and pretreatment procedures (lymphatic nuclear medicine, radiation oncology, and arterial surgery). CONCLUSION ML using baseline administrative data predicts tamoxifen nonadherence. Screening at treatment initiation may support personalized care, improve health outcomes, and minimize cost. Baseline claims may not be sufficient to discriminate adherence. Further validation with enriched longitudinal data may improve model performance.

276 Background: Adherence to tamoxifen among women diagnosed with hormone receptor positive metastatic breast cancer (mBC) can improve survival and minimize recurrence. Screening for non-adherence at treatment initiation may support personalized care, improve health outcomes, and minimize cost of care. This study aimed to use real world data (RWD) and machine learning (ML) methods to classify tamoxifen non-adherence. Methods: A cohort of women diagnosed with incident mBC from 2012 to 2018 were identified from Truven MarketScan Commercial Claims and Encounters and Medicare supplemental administrative claims databases. Patients with < 80% proportion of days coverage (PDC) in the year following treatment initiation were classified non-adherent. Training and internal validation cohorts were randomly generated (4:1 ratio). Clinical procedures, comorbidity, treatment and healthcare encounter features in the year prior to treatment initiation were used to train logistic regression, boosted logistic regression, random forest, and feed forward neural network models and internally validated based on area under receiver operating characteristic (AUROC) curve. The most predictive ML approach was evaluated to assess feature importance. Results: A total of 3,022 patients were included with 39.9% classified as non-adherent. All ML models had moderate predictive accuracy. Logistic regression (AUROC 0.64) was easily interpreted with sensitivity 94% (95% confidence interval [CI]: 0.89, 0.92) and specificity 0.31 (95% CI: 0.29, 0.33). The model accurately classified adherence (negative predictive value 88.7%) but was non-discriminate for non-adherence (positive predictive value 47.7%). Variable importance identified top predictive factors, including patient features (≥55 years old) and pre-treatment procedures (lymphatic nuclear medicine, radiation oncology, arterial surgery). Conclusions: ML using baseline administrative data predicts tamoxifen adherence. Baseline claims may not be sufficient to predict treatment non-adherence. Further validation with enriched longitudinal data may improve model performance for incorporation of predictions into clinical decision support.

PURPOSE Multidisciplinary tumor boards (TBs) are the gold standard for decision-making in cancer care. Variability in preparation, conduction, and impact is widely reported. The benefit of digital technologies to support TBs is unknown. This study evaluated the impact of the NAVIFY Tumor Board solution (NTB) on TB preparation time across multiple user groups in 4 cancer categories: breast, GI, head and neck (ie, ear, nose, and throat, or ENT), and hematopathology. METHODS This prospective study evaluated TB preparation time in multiple phases pre- and post-NTB implementation at an academic health care center. TB preparation times were recorded for multiple weeks using a digital time tracker. RESULTS Preparation times for 59 breast, 61 GI, 36 ENT, and 71 hematopathology cancer TBs comparing a pre-NTB phase to 3 phases of NTB implementation were evaluated between February 2018 and July 2019. NTB resulted in significant reductions in overall preparation time (30%) across 3 TBs pre-NTB compared with the final post-NTB implementation phase. In the breast TB, NTB reduced overall preparation time by 28%, with a 76% decrease in standard deviation (SD). In the GI TB, a 23% reduction in average preparation time was observed for all users, with a 48% decrease in SD. In the ENT TB, a 33% reduction in average preparation time was observed for all users, with a 73% decrease in SD. The hematopathology TB, which was the cocreation partner and initial adopter of the solution, showed variable results. CONCLUSION This study showed a significant impact of a digital solution on time preparation for TBs across multiple users and different TBs, reflecting the generalizability of the NTB. Adoption of such a solution could improve the efficiency of TBs and have a direct economic impact on hospitals.

308 Background: A multidisciplinary tumor board (MTB) provides an interdisciplinary approach for decision-making in cancer care. Efficient conduction of MTBs is importantfor optimal patient management. It is, however, often observed that prepared patient cases are not discussed during tumor boards due to limited time or incomplete information, which could cause delaytocaredecisions and/or the initiation of treatments. It remains unknown whether digital technologies canreduce the rate of failure to discuss during MTBs. Methods: A prospective cohort study was undertaken to evaluate the preparation & conduction of MTBs pre- & post-implementation of the NAVIFY Tumor Board (NTB) solution at Missouri University Health Care (MU), including the Ear, Nose & Throat (ENT) MTB. The NTB is a cloud-based workflow product, integrated with the hospital EMR, that aggregates and displays relevant clinical information. NTB was introduced to theMUENT MTB on Oct 10, 2018. Results: Pre-NTB implementation, data was collected from 42 ENT MTBs. A total of 551 patient cases were prepared for MTBs, but only 423 patient cases were discussed. This was an average “failure-to-discuss” rate of 19.4% per meeting (SD = 15.6%). After NTB implementation, data was collected from 7 MTBs where a total of 70 patient cases were prepared and discussed. There were no instances of failure to discuss, and as such, was significantlyreduced after the implementation of NTB (Mann-Whitney U test, p = 0.0004). The average number of patient cases discussed per meeting pre- and post-NTB implementation did not change (Mann-Whitney U test, p > 0.1) and meeting duration was the same. Conclusions: Introduction of the NTB did not change the weekly number of cases discussed, but did significantly reduce the failure to discuss rates for ENT MTB cases. Reducing failure to discuss rates could decrease the overall time to clinical decision and the initiation of treatment, which could potentially improve patient outcomes. Additional studies are needed to examine the impact of digital solutions on the quality of clinical care.

e18028 Background: A multidisciplinary tumor board (MTB) provides an interdisciplinary approach for decision-making in cancer care. Information factors such as, multiple data sources, incomplete or missing information and teleconferencing failures, have been identified as issues contributing to variability in MTB conduction and impact. Little is known about how digital tumor board solutions can optimize MTB meeting conduction. Methods: A prospective IRB approved cohort study was undertaken to evaluate the time for patient case discussions,before and after the implementation of the NAVIFY Tumor Board (NTB) solution, at University of Missouri Health Care. Data was collected using a digital time-tracking application. The NTB manual version was implemented via a phased roll-out (Breast May 18, 2018; Gastrointestinal (GI) Jul 11, 2018; ENT Oct 30, 2018 – no manual version phase). Subsequently, the integrated version was introduced (Oct 4, 2018) whereby automated electronic medical record (EMR) data extraction was enabled. Results: Patient case discussion time was recorded at 138 MTBs (1109 patient cases) during 2018 (Breast 40 MTB; 236cases / GI 49 MTB; 389 cases / ENT 49 MTBs; 484 cases). Case discussion time significantly reduced at the Breast MTB (6.6mins to 5.3 mins; p-value = 0.01). Case discussion time at ENT MTB and GI MTB showed no significant change (Table 1). Time variance in case discussions significantly decreased post-NTB implementation at the Breast MTB (p-value = 0.008). For the GI & ENT MTBs, there was no significant difference post-NTB implementation (GI p-value =0.199; ENT p-value = 0.511), however, variance was already sma Conclusions: Introduction of the NTB reduced the time spent discussing cases for the Breast MTB, but showed no change forGI MTB and ENT MTB. Interestingly, case discussion times converged to between 5.5 - 6.5 minutesirrespective of the MTB type. Furthermore, variance decreased or was already small, for all MTB types. Both observations suggest that NTB standardized the case discussion process, perhaps because it created a common format for case presentation. A common workflow tool for MTB meeting conduction could ensure availability of all required data, increase efficiency of therapy decision-making, and lead to higher throughput of cases resulting in shorter time-to-treatment.

e18020 Background: A multidisciplinary tumor board (MTB) provides an interdisciplinary approach for decision-making in cancer care. Oncology nurse navigators (NN) have been introduced to facilitate patient access to services and resources, and to improve continuity and coordination of care. Little is known about how digital technologies support this role. Methods: A prospective IRB approved cohort study was undertaken to evaluate NN time preparation for MTBs before and after the implementation of the NAVIFY Tumor Board (NTB) solution at University of Missouri Health Care (MU). Data was collected using a digital time-tracking application. The NTB manual version was implemented via a phased roll-out (Hematology – Apr 9, 2018; Breast May 18, 2018; Gastrointestinal Jul 11, 2018). Subsequently, the integrated version was introduced (Oct 4, 2018). Results: Time preparation for 101 MTBs (828 cases) were evaluated throughout 2018 (35 Breast; 213 cases / 32 GI; 302 cases / 34 Hematology; 313 cases). Data showed NN meeting preparation time decreased for each MTB after the introduction of NTB; further time saving was achieved with NTB integrated version (Table 1). For the Breast MTB there was statistical significance between time preparation pre- & post-NTB (p-value = 0.011) but not between the manual and integrated versions (p-value = 0.118). For the Gastrointestinal MTB, there was no statistical significance between pre- & post-NTB (p-value = 0.214). For Hematology MTB there was statistical significance between the change from the NTB manual version vs. integration (p-value = 0.004). Note before NTB implementation Hematology MTB was not supported by NN. Conclusions: Introduction of the NTB reduced time spent by NNs preparing for 3 different MTBs, EMR integration delivered further time-savings. No discernable learning curve was observed suggesting user-training was sufficient and NTB was easy to learn. A reduction in variance (SD & IQR) for time preparation across all MTBs was seen. This suggests the NTB solution standardized the process for MTB preparation. Less time spent preparing for MTBs gives NNs more time to support patients.