Background: Medication harm affects between 5 and 15% of hospitalised patients, with approximately half of the harm events considered preventable through timely intervention. The Adverse Inpatient Medication Event (AIME) risk prediction model was previously developed to guide a systematic approach to patient prioritisation for targeted clinician review, but frailty was not tested as a candidate predictor variable. Aim: To evaluate the predictive performance of an updated AIME model, incorporating a measure of frailty, when applied to a new multisite cohort of hospitalised adult inpatients. Methods: A retrospective cohort study was conducted at two tertiary Australian hospitals on patients discharged between 1st January and April 31, 2020. Data were extracted from electronic medical records (EMRs) and clinical coding databases. Medication harm was identified using ICD-10 Y -codes and confirmed by senior pharmacist review of medical records. The Hospital Frailty Risk Score (HFRS) was calculated for each patient. Logistic regression analysis was used to construct a modified AIME model. Candidate variables of the original AIME model, together with new variables including HFRS were tested. Performance of the final model was reported using area under the curve (AUC) and decision curve analysis (DCA). Results: A total of 4089 patient admissions were included, with a mean age +/- standard deviation (SD) of 64 years (+/- 19 years), 2050 patients (50%) were males, and mean HFRS was 6.2 (+/- 5.9). 184 patients (4.5%) experienced one or more medication harm events during hospitalisation. The new AIME-Frail risk model incorporated 5 of the original variables: length of stay (LOS), anti -psychotics, antiarrhythmics, immunosuppressants, and INR greater than 3, as well as 5 new variables: HFRS, anticoagulants, antibiotics, insulin, and opioid use. The AUC was 0.79 (95% CI: 0.76-0.83) which was superior to the original model (AUC = 0.70, 95% CI: 0.65-0.74) with a sensitivity of 69%, specificity of 81%, positive predictive value of 0.14 (95% CI: 0.10-0.17) and negative predictive value of 0.98 (95% CI: 0.97-0.99). The DCA identified the model as having potential clinical utility between the probability thresholds of 0.05-0.4. Conclusion: The inclusion of a frailty measure improved the predictive performance of the AIME model. Screening inpatients using the AIME-Frail tool could identify more patients at high -risk of medication harm who warrant timely clinician review.

Background: Effective fall prevention interventions in hospitals require appropriate allocation of resources early in admission.To address this, fall risk prediction tools and models have been developed with the aim to provide fall prevention strategies topatients at high risk. However, fall risk assessment tools have typically been inaccurate for prediction, ineffective in prevention,and time-consuming to complete. Accurate, dynamic, individualized estimates of fall risk for admitted patients using routinelyrecorded data may assist in prioritizing fall prevention efforts.Objective: The objective of this study was to develop and validate an accurate and dynamic prognostic model for inpatient fallsamong a cohort of patients using routinely recorded electronic medical record data.Methods: We used routinely recorded data from 5 Australian hospitals to develop and internally-externally validate a predictionmodel for inpatient falls using a Cox proportional hazards model with time-varying covariates. The study cohort included patientsadmitted during 2018-2021 to any ward, with no age restriction. Predictors used in the model included admission-relatedadministrative data, length of stay, and number of previous falls during the admission (updated every 12 hours up to 14 days afteradmission). Model calibration was assessed using Poisson regression and discrimination using the area under the time-dependentreceiver operating characteristic curve.Results: There were 1,107,556 inpatient admissions, 6004 falls, and 5341 unique fallers. The area under the time-dependentreceiver operating characteristic curve was 0.899 (95% CI 0.88-0.91) at 24 hours after admission and declined throughout admission(eg, 0.765, 95% CI 0.75-0.78 on the seventh day after admission). Site-dependent overestimation and underestimation of riskwas observed on the calibration plots.Conclusions: Using a large dataset from multiple hospitals and robust methods to model development and validation, wedeveloped a prognostic model for inpatient falls. It had high discrimination, suggesting the model has the potential foroperationalization in clinical decision support for prioritizing inpatients for fall prevention. Performance was site dependent, andmodel recalibration may lead to improved performance.

Objective:To describe development and application of a checklist of criteria for selecting an automated machine learning (Auto ML) platform for use in creating clinical ML models.Materials and Methods:Evaluation criteria for selecting an Auto ML platform suited to ML needs of a local health district were developed in 3 steps: (1) identification of key requirements, (2) a market scan, and (3) an assessment process with desired outcomes.Results:The final checklist comprising 21 functional and 6 non-functional criteria was applied to vendor submissions in selecting a platform for creating a ML heparin dosing model as a use case.Discussion:A team of clinicians, data scientists, and key stakeholders developed a checklist which can be adapted to ML needs of healthcare organizations, the use case providing a relevant example.Conclusion:An evaluative checklist was developed for selecting Auto ML platforms which requires validation in larger multi-site studies.

A government performance audit is an independent evaluation of a government entity's activities and operations aimed at improving its efficiency, effectiveness, and accountability.Audit offices are frequently facing the challenge of selecting an audit topic for different government sectors that justifies the use of public money to conduct the performance audit.Text mining techniques have been rarely mentioned in association with selecting performance audit topics in the literature.In this work, we identify potential performance audit topics using topic modelling, an unsupervised machine learning approach.Topic modelling has been employed to create a demonstration system aimed at showcasing the utility of text mining tools in identifying potential audit topics.The system does not actually choose topics for auditors, it is a decision support tool that displays to them the distribution of topics and related keywords across a set of processed documents.Auditors must still analyse and interpret the results from the text mining system to make informed decisions.This work also includes a document retrieval system, enabling further exploration of specific topics.The outcome of this study suggests that incorporating text mining in the stage of identifying performance audit topics will streamline the topic selection process and decrease the amount of time required for manual information gathering at the outset.

BackgroundEffective fall prevention interventions in hospitals require appropriate allocation of resources early in admission. To address this, fall risk prediction tools and models have been developed with the aim to provide fall prevention strategies to patients at high risk. However, fall risk assessment tools have typically been inaccurate for prediction, ineffective in prevention, and time-consuming to complete. Accurate, dynamic, individualized estimates of fall risk for admitted patients using routinely recorded data may assist in prioritizing fall prevention efforts. ObjectiveThe objective of this study was to develop and validate an accurate and dynamic prognostic model for inpatient falls among a cohort of patients using routinely recorded electronic medical record data. MethodsWe used routinely recorded data from 5 Australian hospitals to develop and internally-externally validate a prediction model for inpatient falls using a Cox proportional hazards model with time-varying covariates. The study cohort included patients admitted during 2018-2021 to any ward, with no age restriction. Predictors used in the model included admission-related administrative data, length of stay, and number of previous falls during the admission (updated every 12 hours up to 14 days after admission). Model calibration was assessed using Poisson regression and discrimination using the area under the time-dependent receiver operating characteristic curve. ResultsThere were 1,107,556 inpatient admissions, 6004 falls, and 5341 unique fallers. The area under the time-dependent receiver operating characteristic curve was 0.899 (95% CI 0.88-0.91) at 24 hours after admission and declined throughout admission (eg, 0.765, 95% CI 0.75-0.78 on the seventh day after admission). Site-dependent overestimation and underestimation of risk was observed on the calibration plots. ConclusionsUsing a large dataset from multiple hospitals and robust methods to model development and validation, we developed a prognostic model for inpatient falls. It had high discrimination, suggesting the model has the potential for operationalization in clinical decision support for prioritizing inpatients for fall prevention. Performance was site dependent, and model recalibration may lead to improved performance.

The clinical prioritisation criteria (CPC) are a clinical decision support tool that ensures patients referred for public specialist outpatient services to Queensland Health are assessed according to their clinical urgency. Medical referrals are manually triaged and prioritised into three categories by the associated health service before appointments are booked. We have developed a method using artificial intelligence to automate the process of categorizing medical referrals based on clinical prioritization criteria (CPC) guidelines. Using machine learning techniques, we have created a tool that can assist clinicians in sorting through the substantial number of referrals they receive each year, leading to more efficient use of clinical specialists' time and improved access to healthcare for patients. Our research included analyzing 17,378 ENT referrals from two hospitals in Queensland between 2019 and 2022. Our results show a level of agreement between referral categories and generated predictions of 53.8%.

Background The recent digitisation and uptake of integrated electronic medical records (ieMR) within hospital systems has provided opportunities to improve the performance and usefulness of inpatient fall risk prediction models, including more frequently updated risk estimates for patients throughout their admission. Aims We aimed to develop and internally validate a prediction model for inpatient falls using data obtained from the ieMR. Methods We extracted data from the Princess Alexandra Hospital ieMR and associated fall events from RiskMan for all admissions in 2019 with a length of stay greater than 24 hours. Data extracted included ADL and cognitive assessments, vital signs, procedures, medications, hospital ward, and patient demographics. Internal validation was performed by cross validation. Predictions were made every 24 hours throughout the admission and the prediction target were falls that occurred within the 24 hours after the time of prediction. Three modelling approaches were assessed: random forest, LightGBM, and logistic regression. Results There were 1,160 falls that occurred during 34,434 admissions (280,898 patient-days). Fallers were older than non-fallers (mean ages of 65.4 and 58.8 years, respectively). Preliminary findings indicate the LightGBM model had the best discrimination with an area under the receiver operator characteristic curve of 0.631 (95% CI: [0.617 to 0.645]). Conclusion This novel Australian study demonstrated that falls are hard to predict accurately, even with detailed data. Further refinements are planned. Learning Outcomes This presentation will provide practical insights into the use of ieMR data for the development of contemporary fall prediction models.

To externally evaluate the performance of two European risk prediction models, for identifying patients at high-risk of medication harm, in an Australian hospital setting. This was a secondary analysis of a pre-existing cohort study described in a recently published study by Falconer et al. (Br J Clin Pharmacol 87(3):1512–1524, 2021) describing the development of a predictive risk model for inpatient medication harm. We retrospectively extracted relevant variables using the electronic health records of general medical and geriatric patients admitted to a quaternary hospital, in Brisbane, over 6 months from July to December 2017. This dataset was used to externally evaluate the two European models, The Brighton Adverse Drug Reaction Risk (BADRI) model by Tangiisuran et al. and a risk model developed by Trivalle et al. The variables were entered into both models and the patients’ risk of medication harm was calculated, and compared with actual patient outcomes. Predictive performance was evaluated by measuring area under the receiver operative characteristic (AuROC) curves. The Australian patient cohort included 1982 patients (median age 74 years), of which 136 (7%) patients experienced ≥ 1 medication harm event(s). External evaluation of the two European models identified that both the BADRI and the Trivalle models had reduced predictive performance in an Australian patient cohort, compared with their original studies (AuROC of 0.63 [95% CI: 0.58–0.68] and 0.60 [95% CI: 0.55–0.65], respectively). Neither model demonstrated sufficient discrimination to warrant further evaluation in our local setting. This is likely a result of variations between the development and the validation cohorts, and the change in healthcare systems over time, and highlights the need for an up-to-date and context-specific risk prediction model.

Background Unfractionated heparin (UFH) is an anticoagulant drug that is considered a high-risk medication because an excessive dose can cause bleeding, whereas an insufficient dose can lead to a recurrent embolic event. Therapeutic response to the initiation of intravenous UFH is monitored using activated partial thromboplastin time (aPTT) as a measure of blood clotting time. Clinicians iteratively adjust the dose of UFH toward a target, indication-defined therapeutic aPTT range using nomograms, but this process can be imprecise and can take ≥36 hours to achieve the target range. Thus, a more efficient approach is required. Objective In this study, we aimed to develop and validate a machine learning (ML) algorithm to predict aPTT within 12 hours after a specified bolus and maintenance dose of UFH. Methods This was a retrospective cohort study of 3019 patient episodes of care from January 2017 to August 2020 using data collected from electronic health records of 5 hospitals in Queensland, Australia. Data from 4 hospitals were used to build and test ensemble models using cross-validation, whereas data from the fifth hospital were used for external validation. We built 2 ML models: a regression model to predict the aPTT value after a UFH bolus dose and a multiclass model to predict the aPTT, classified as subtherapeutic (aPTT <70 seconds), therapeutic (aPTT 70-100 seconds), or supratherapeutic (aPTT >100 seconds). Modeling was performed using Driverless AI (H2O), an automated ML tool, and 17 different experiments were iteratively conducted to optimize model accuracy. Results In predicting aPTT, the best performing model was an ensemble with 4x LightGBM models with a root mean square error of 31.35 (SD 1.37). In predicting the aPTT class using a repurposed data set, the best performing ensemble model achieved an accuracy of 0.599 (SD 0.0289) and an area under the receiver operating characteristic curve of 0.735. External validation yielded similar results: root mean square error of 30.52 (SD 1.29) for the aPTT prediction model, and accuracy of 0.568 (SD 0.0315) and area under the receiver operating characteristic curve of 0.724 for the aPTT multiclassification model. Conclusions To the best of our knowledge, this is the first ML model applied to intravenous UFH dosing that has been developed and externally validated in a multisite adult general medical and surgical inpatient setting. We present the processes of data collection, preparation, and feature engineering for replication.

The Electronic Medical Record (EMR) provides an opportunity to manage patient care efficiently and accurately. This includes clinical decision support tools for the timely identification of adverse events or acute illnesses preceded by deterioration. This paper presents a machine learning-driven tool developed using real-time EMR data for identifying patients at high risk of reaching critical conditions that may demand immediate interventions. This tool provides a pre-emptive solution that can help busy clinicians to prioritize their efforts while evaluating the individual patient risk of deterioration. The tool also provides visualized explanation of the main contributing factors to its decisions, which can guide the choice of intervention. When applied to a test cohort of 18,648 patient records, the tool achieved 100% sensitivity for prediction windows 2-8 h in advance for patients that were identified at 95%, 85% and 70% risk of deterioration.

Aim To identify and critically appraise studies of prediction models, developed using machine learning (ML) methods, for determining the optimal dosing of unfractionated heparin (UFH). Methods Embase, PubMed, CINAHL, Web of Science, International Pharmaceutical Abstracts and IEEE Xplore databases were searched from inception to 31 January 2020 to identify relevant studies using key search terms synonymous with artificial intelligence or ML, ‘prediction’, ‘dose’, ‘activated partial thromboplastin time (aPTT)’ and ‘UFH.’ Studies had to have used ML methods for developing models that predicted optimal dose of UFH or target therapeutic aPTT levels in the hospital setting. The CHARMS Checklist was used to assess quality and risk of bias of included studies. Results Of 8393 retrieved abstracts, 61 underwent full text review and eight studies met inclusion criteria. Four studies described models for predicting aPTT, three studies described models predicting optimal dose of heparin during dialysis and one study described a model that used surrogate outcomes of clotting and bleeding to predict a therapeutic aPTT. Studies varied widely in reporting of study participants, feature characterisation and selection, handling of missing data, sample size calculations and the intended clinical application of the model. Only one study conducted an external validation and no studies evaluated model impacts in clinical practice. Conclusion Studies of ML models for UFH dosing are few and none report a model ready for routine clinical use. Existing studies are limited by low methodological quality, inadequate reporting of study factors and absence of external validation and impact analysis.

The clinical nursing and midwifery dashboard (CNMD) was built to provide a near real-time information and data visualisations for nurse unit managers (NUMs) and maternity unit managers (MUMs) within only a 5-15 minutes delay from when they enter data to the integrated electronic medical records (ieMR) system. The dashboard displays metrics and information about current adult inpatients in overnight wards. The aim is to support NUMs and MUMs to manage their daily workload and have continuous visibility of patients nursing risk and safety assessment documentation. A quantitative evaluation approach was conducted to measure the impact of the dashboard on key performance indicators. Statistical analysis was completed to compare risk assessment average completion times prior to and post CNMD implementation. The results of the evaluation were positive, and the statistical analysis shows significant reduction in the average time to complete different risk assessments with p-value<0.01.

Artificial intelligence (AI) has become a mainstream technology in many industries, but not yet in health care. Although basic research and commercial investment are burgeoning across various clinical disciplines, AI remains relatively non-existent in most healthcare organisations. This is despite hundreds of AI applications having passed proof-of-concept phase, and scores receiving regulatory approval overseas. AI has considerable potential to optimise multiple care processes, maximise workforce capacity, reduce waste and costs, and improve patient outcomes. The current obstacles to wider AI adoption in health care and the pre-requisites for its successful development, evaluation and implementation need to be defined.

Manual theatre performance measurement is resource yearning and inaccurate. To automate the process, we built a dashboard which provides interactive visualisation of key performance metrics related to operating theatres. The aim is to assist in the efficient management of surgical services and provide visibility on metrics trending over time for health service facilities.

Unfractionated heparin (UFH), is an anticoagulant drug considered a high-risk medication in that an excessive dose can cause bleeding, while an insufficient dose can lead to a recurrent embolic event. Following initiation of intravenous (IV) UFH, the therapeutic response is monitored using a measure of blood clotting time known as the activated partial thromboplastin time (aPTT). Clinicians iteratively adjust the dose of UFH to a target aPTT range, with the usual therapeutic target range between 60 to 100 seconds. The aim of this study was to develop and validate a ML algorithm to predict, aPTT within 12 hours after a specified bolus and maintenance dose of UFH. This was a retrospective cohort study of 3273 episodes of care from January 2017 to August 2020 using data collected from electronic health records (EHR) of five hospitals in Queensland, Australia. Data from four hospitals were used to build and test ensemble models using cross validation, while the data from the fifth hospital was used for external validation. Modelling was performed using H2O Driverless AI® an automated ML tool, and 17 different experiments were conducted in an iterative process to optimise model accuracy. In predicting aPTT, the best performing experiment produced an ensemble with 4x LightGBM models with a root mean square error (RMSE) of 31.35. This dataset was re-purposed as a multi-classification task (sub-therapeutic, therapeutic, and supra-therapeutic aPTT result) and achieved a 59.9% accuracy and area under the receiver operating characteristic curve (AUC) of 0.735. External validation yielded similar results: RMSE of 30.52 +/- 1.29 for the prediction model, and accuracy of 56.8% +/- 3.15 and AUC of 0.724 for the multi-classification model. According to our knowledge, this is the first study of ML applied to IV UFH dosing that has been developed and externally validated in a multisite adult general medical inpatient setting. We present the processes of data collection, preparation, and feature engineering for purposes of replication.

AimsMedication harm has negative clinical and economic consequences, contributing to hospitalisation, morbidity and mortality. The incidence ranges from 4 to 14%, of which up to 50% of events may be preventable. A predictive model for identifying high‐risk inpatients can guide a timely and systematic approach to prioritisation. The aim of this study is to develop and internally validate a risk prediction model for prioritisation of hospitalised patients at risk of medication harm.MethodsA retrospective cohort study was conducted in general medical and geriatric specialties at an Australian hospital over six months. Medication harm was identified using International Classification of Disease (ICD‐10) codes and the hospital's incident database. Sixty‐eight variables, including medications and laboratory results, were extracted from the hospital's databases. Multivariable logistic regression was used to develop the final risk model. Performance was evaluated using area under the receiver operative characteristic curve (AuROC) and clinical utility was determined using decision curve analysis.ResultsThe study cohort included 1982 patients with median age 74 years, of which 136 (7%) experienced at least one adverse medication event(s). The model included: length of stay, hospital re‐admission within 12 months, venous or arterial thrombosis and/or embolism, ≥ 8 medications, serum sodium < 126 mmol/L, INR > 3, anti‐psychotic, antiarrhythmic and immunosuppressant medications, and history of medication allergy. Validation gave an AuROC of 0.70 (95% CI: 0.65–0.74). Decision curve analysis identified that the AIME may be clinically useful to help guide decision making in practice.ConclusionWe have developed a predictive model with reasonable performance. Future steps include external validation and impact evaluation.

With the extensive use of rating systems in the web, and their significance in decision making process by users, the need for more accurate aggregation methods has emerged. The Naive aggregation method, using the simple mean, is not adequate anymore in providing accurate reputation scores for items [6 ], hence, several researches where conducted in order to provide more accurate alternative aggregation methods. Most of the current reputation models do not consider the distribution of ratings across the different possible ratings values. In this paper, we propose a novel reputation model, which generates more accurate reputation scores for items by deploying the normal distribution over ratings. Experiments show promising results for our proposed model over state-of-the-art ones on sparse and dense datasets.

Reputation systems are employed to measure the quality of items on the Web. Incorporating accurate reputation scores in recommender systems is useful to provide more accurate recommendations as recommenders are agnostic to reputation. The ratings aggregation process is a vital component of a reputation system. Reputation models available do not consider statistical data in the rating aggregation process. This limitation can reduce the accuracy of generated reputation scores. In this paper, we propose a new reputation model that considers previously ignored statistical data. We compare our proposed model against state-of the-art models using top-N recommender system experiment.

Many websites presently provide the facility for users to rate items quality based on user opinion. These ratings are used later to produce item reputation scores. The majority of websites apply the mean method to aggregate user ratings. This method is very simple and is not considered as an accurate aggregator. Many methods have been proposed to make aggregators produce more accurate reputation scores. In the majority of proposed methods the authors use extra information about the rating providers or about the context (e.g. time) in which the rating was given. However, this information is not available all the time. In such cases these methods produce reputation scores using the mean method or other alternative simple methods. In this paper, we propose a novel reputation model that generates more accurate item reputation scores based on collected ratings only. Our proposed model embeds statistical data, previously disregarded, of a given rating dataset in order to enhance the accuracy of the generated reputation scores. In more detail, we use the Beta distribution to produce weights for ratings and aggregate ratings using the weighted mean method. Experiments show that the proposed model exhibits performance superior to that of current state-of-the-art models.