ABSTRACT Evaluating synthetic data produced by generative models remains a critical challenge in sensitive domains such as healthcare and finance. Ensuring that such data is ‘faithful’ to real data is essential for downstream applications and decision‐making, including regulatory compliance. This paper introduces an AI‐powered interactive visual system—AIS yst —designed to assess the fidelity of synthetic tabular datasets. The system supports multilevel comparisons with real datasets, spanning multivariate resemblance analyses based on dimensionality reduction through suitable two‐dimensional projections, bivariate correlation and univariate similarity. AIS yst also integrates an AI assistant by leveraging state‐of‐the‐art large language models to summarize key findings and generate suggestions for improving synthetic data generation models. We validated the capabilities of AIS yst through three case studies, supported by feedback from industrial AI experts who endorsed its broader deployment.

Objective High-quality ethnicity data are necessary for tackling health inequalities. This study aims to support the improvement of ethnicity recording in emergency care data by (a) investigating variation in unknown ethnicity records, (b) exploring patterns of missingness between ethnicity and other variables and (c) identifying the variables most important in the recording of an unknown ethnicity.Design Secondary analysis of the Emergency Care Data Set (ECDS).Setting and participants National Health Service (NHS) and independent sector organisations who provide emergency care services in England. Includes all anonymised ECDS records from the financial year 2023/2024, excluding those with a ‘Null’ anonymised pseudo-NHS number or those patients announced as dead on arrival (24 167 154 records after exclusions).Results Differences in the percentage of unknown ethnicity records were seen across ages, organisation types and attendance characteristics. An increase in the percentage of missing values was observed for 62 variables when ethnicity was unknown, with the largest increases shown by variables related to investigations and treatments received by the patient (15.31% and 14.74%). Provider site code was identified as the most important variable in recording ethnicity as unknown. Further analysis highlighted a subset of acute trusts and independent sector organisations that disproportionately contributed to unknown ethnicity records.Conclusions This study has improved understanding of unknown ethnicity recording by exploring variation across demographics and organisation types and demonstrating an increase in missingness across other ECDS variables when ethnicity is unknown. Unknown ethnicity recording appears to be a site-specific problem, which should be addressed through improvements to the data collection processes of individual providers. These insights should be used to improve the quality of ethnicity coding within emergency care data, necessary for tackling health inequalities.

Scatterplots are widely used across various domains to identify anomalies in datasets, particularly in multi-class settings, such as detecting misclassified or mislabeled data. However, scatterplot effectiveness often declines with large datasets due to limited display resolution. This paper introduces a novel Visual Quality Measure (VQM) - OM4AnI (Overlap Measure for Anomaly Identification) - which quantifies the degree of overlap for identifying anomalies, helping users estimate how effectively anomalies can be observed in multi-class scatterplots. OM4AnI begins by computing anomaly index based on each data point's position relative to its class cluster. The scatterplot is then discretized into a matrix representation by binning the display space into cell-level (pixel-level) grids and computing the coverage for each pixel. It takes into account the anomaly index of data points covering these pixels and visual features (marker shapes, marker sizes, and rendering orders). Building on this foundation, we sum all the coverage information in each cell (pixel) of matrix representation to obtain the final quality score with respect to anomaly identification. We conducted an evaluation to analyze the efficiency, effectiveness, sensitivity of OM4AnI in comparison with six representative baseline methods that are based on different computation granularity levels: data level, marker level, and pixel level. The results show that OM4AnI outperforms baseline methods by exhibiting more monotonic trends against the ground truth and greater sensitivity to rendering order, unlike the baseline methods. It confirms that OM4AnI can inform users about how effectively their scatterplots support anomaly identification. Overall, OM4AnI shows strong potential as an evaluation metric and for optimizing scatterplots through automatic adjustment of visual parameters.

Datasets often have missing values. Understanding the locations and underlying reasons (missing data patterns and structures, respectively) is important to avoid analysis bias or building inappropriate models. However, investigating missing data patterns and structures presents significant challenges. To address them we developed a novel visual analytic method, which combines new purity metrics for three core types of pattern (block, monotone and disjoint) with seven different visualizations in an iterative workflow. We also conducted two case studies, one with a UCI Machine Learning Repository dataset and the other with a 21 million record/389 variable hospital dataset that had many interwoven missingness patterns. In the UCI case study, the method's purity metrics and heatmaps revealed a rare data quality issue that affected 0.08% of records, but was not apparent when an existing correlation-based approach was used. Our method also implements perceptual discontinuity to ensure that small values are visible in bar charts, and that showed that two variables were missing values in 0.02% and 0.3% of records, respectively, rather than being complete as reported in the original UCI paper. The hospital dataset contained 595,171 unique combinations of missing values, but that overwhelming number was reduced to 57 comprehensible patterns by iteratively using purity metrics and heatmaps with the three core pattern types in our method. That revealed insights that ranged from uncovering rare and common data quality issues, to finding data dictionary errors, and making explicit relationships between groups of variables that may otherwise be hidden. The method also provides an explanation graph, which aids reproducibility by documenting the order in which patterns were found and acts as a tangible artifact that should help communicate the findings to stakeholders.

BACKGROUND:Structured medication reviews (SMRs) are an essential component of medication optimization, especially for patients with multimorbidity and polypharmacy. However, the process remains challenging due to the complexities of patient data, time constraints, and the need for coordination among health care professionals (HCPs). This study explores HCPs' perspectives on the integration of artificial intelligence (AI)-assisted tools to enhance the SMR process, with a focus on the potential benefits of and barriers to adoption. OBJECTIVE:This study aims to identify the key user requirements for AI-assisted tools to improve the efficiency and effectiveness of SMRs, specifically for patients with multimorbidity, complex polypharmacy, and frailty. METHODS:A qualitative study was conducted involving focus groups and semistructured interviews with HCPs and patients in the United Kingdom. Participants included physicians, pharmacists, clinical pharmacologists, psychiatrists from primary and secondary care, a policy maker, and patients with multimorbidity. Data were analyzed using a hybrid inductive and deductive thematic analysis approach to identify themes related to AI-assisted tool functionality, workflow integration, user-interface visualization, and usability in the SMR process. RESULTS:Four major themes emerged from the analysis: innovative AI potential, optimizing electronic patient record visualization, functionality of the AI tool for SMRs, and facilitators of and barriers to AI tool implementation. HCPs identified the potential of AI to support patient identification and prioritizing those at risk of medication-related harm. AI-assisted tools were viewed as essential in detecting prescribing gaps, drug interactions, and patient risk trajectories over time. Participants emphasized the importance of presenting patient data in an intuitive format, with a patient interface for shared decision-making. Suggestions included color-coding blood results, highlighting critical medication reviews, and providing timelines of patient medical histories. HCPs stressed the need for AI tools to integrate seamlessly with existing electronic patient record systems and provide actionable insights without overwhelming users with excessive notifications or "pop-up" alerts. Factors influencing the uptake of AI-assisted tools included the need for user-friendly design, evidence of tool effectiveness (though some were skeptical about the predictive accuracy of AI models), and addressing concerns around digital exclusion. CONCLUSIONS:The findings highlight the potential for AI-assisted tools to streamline and optimize the SMR process, particularly for patients with multimorbidity and complex polypharmacy. However, successful implementation depends on addressing concerns related to workflow integration, user acceptance, and evidence of effectiveness. User-centered design is crucial to ensure that AI-assisted tools support HCPs in delivering high-quality, patient-centered care while minimizing cognitive overload and alert fatigue.

Codelists play a crucial role in ensuring accurate and standardized communication within healthcare. However, preparation of high-quality codelists is a rigorous and time-consuming process. The literature focuses on transparency of clinical codelists and overlooks the utility of automation. Here we present a Codelist Generation Framework that can automate generation of codelists with minimal input from clinical experts. We demonstrate the process using a specific project, DynAIRx, producing appropriate codelists and a framework allowing future projects to take advantage of automated codelist generation. Both the framework and codelist are publicly available. DynAIRx is an NIHR-funded project aiming to develop AIs to help optimise prescribing of medicines in patients with multiple long-term conditions. DynAIRx requires complex codelists to describe the trajectory of each patient, and the interaction between their conditions. We promptly generated ≈ 214 codelists for DynAIRx using the proposed framework and validated them with a panel of experts, significantly reducing the amount of time required by making effective use of automation. The framework reduced the clinician time required to validate codes, automatically shrunk codelists using trusted sources and added new codes for review against existing codelists. In the DynAIRx case study, a codelist of ≈ 14000 codes required only 7-9 hours of clinician’s time in the end (while existing methods takes months), and application of the automation framework reduced the workload by >80

Dashboards have arguably been the most used visualizations during the COVID-19 pandemic. They were used to communicate its evolution to national governments for disaster mitigation, to the public domain to inform about its status, and to epidemiologists to comprehend and predict the evolution of the disease. Each design had to be tailored for different tasks and to varying audiences—in many cases set up in a very short time due to the urgent need. In this article, we collect notable examples of dashboards and reflect on their use and design during the pandemic from a user-oriented perspective. We interview a group of researchers with varying visualization expertise who actively used dashboards during the pandemic as part of their daily workflow. We discuss our findings and compile a list of lessons learned to support future visualization researchers and dashboard designers.

Background: Population ageing has led to an increase in multimorbidity and polypharmacy. Some medications may need to be stopped, but patient attitudes towards deprescribing are poorly understood. This study explores attitudes towards (de)prescribing in patients with multimorbidity in the UK primary care. Methods: Patients with multimorbidity were invited to complete the Revised Patients Attitudes Towards Deprescribing (rPATD) Questionnaire using the Evergreen Life Personal Health Record App (Manchester, UK). The responses were linked to electronic health records. Anonymised data were analysed in a trusted research environment (University of Liverpool) for group comparisons and using multivariable logistic regression to identify factors associated with satisfaction with current medications. Results: A total 1,019 patients participated in the study (n=365 aged <65, 30% males; n=654 ≥65, 57% males). Most patients were satisfied with their current medications (74% aged <65, 70% aged ≥65) but were willing to stop one or more of their regular medicines if their doctor said it was possible (82%, 68% accordingly). Polypharmacy, use of antihypertensive drugs, and antidepressants were associated with patient-reported burden in taking medicines. Frailty did not influence patient deprescribing attitudes. Patients who were satisfied with current medications had fewer medications. Independent predictors of satisfaction with current medications were higher total involvement and appropriateness scores, and lower total burden score. Conclusions: Most patients with multimorbidity would consider stopping some of their medications, even when they are generally satisfied with the treatments they received. Frailty status does not imply willingness to stop medications. Clinicians should discuss medication deprescribing for shared decision. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This study/project is funded by the National Institute for Health Research (NIHR) under its Programme Artificial Intelligence for Multiple and Long-Term Conditions (NIHR203986). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. IB is supported by NIHR as Senior Investigator award (NIHR205131).AW is partly funded by Health and Care Research Wales award (NHS-RTA-21-02) ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The Newcastle North Tyneside Research Ethics Committee (REC reference:22/NE/0088) granted ethical approval for the DynAIRx study. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced in the present study are available upon reasonable request to the authors

IntroductionStructured medication reviews (SMRs), introduced in the United Kingdom (UK) in 2020, aim to enhance shared decision-making in medication optimisation, particularly for patients with multimorbidity and polypharmacy. Despite its potential, there is limited empirical evidence on the implementation of SMRs, and the challenges faced in the process. This study is part of a larger DynAIRx (Artificial Intelligence for dynamic prescribing optimisation and care integration in multimorbidity) project which aims to introduce Artificial Intelligence (AI) to SMRs and develop machine learning models and visualisation tools for patients with multimorbidity. Here, we explore how SMRs are currently undertaken and what barriers are experienced by those involved in them.MethodsQualitative focus groups and semi-structured interviews took place between 2022-2023. Six focus groups were conducted with doctors, pharmacists and clinical pharmacologists (n = 21), and three patient focus groups with patients with multimorbidity (n = 13). Five semi-structured interviews were held with 2 pharmacists, 1 trainee doctor, 1 policy-maker and 1 psychiatrist. Transcripts were analysed using thematic analysis.ResultsTwo key themes limiting the effectiveness of SMRs in clinical practice were identified: 'Medication Reviews in Practice' and 'Medication-related Challenges'. Participants noted limitations to the efficient and effectiveness of SMRs in practice including the scarcity of digital tools for identifying and prioritising patients for SMRs; organisational and patient-related challenges in inviting patients for SMRs and ensuring they attend; the time-intensive nature of SMRs, the need for multiple appointments and shared decision-making; the impact of the healthcare context on SMR delivery; poor communication and data sharing issues between primary and secondary care; difficulties in managing mental health medications and specific challenges associated with anticholinergic medication.ConclusionSMRs are complex, time consuming and medication optimisation may require multiple follow-up appointments to enable a comprehensive review. There is a need for a prescribing support system to identify, prioritise and reduce the time needed to understand the patient journey when dealing with large volumes of disparate clinical information in electronic health records. However, monitoring the effects of medication optimisation changes with a feedback loop can be challenging to establish and maintain using current electronic health record systems.

The use of good-quality data to inform decision making is entirely dependent on robust processes to ensure it is fit for purpose. Such processes vary between organisations, and between those tasked with designing and following them. In this article we report on a survey of 53 data analysts from many industry sectors, 24 of whom also participated in in-depth interviews, about computational and visual methods for characterizing data and investigating data quality. The paper makes contributions in two key areas. The first is to data science fundamentals, because our lists of data profiling tasks and visualization techniques are more comprehensive than those published elsewhere. The second concerns the application question "what does good profiling look like to those who routinely perform it?", which we answer by highlighting the diversity of profiling tasks, unusual practice and exemplars of visualization, and recommendations about formalizing processes and creating rulebooks.

Using either a search engine or website navigation can pose difficulties for web users in certain situations. Therefore, creating a useful visualization of the website structure can aid web users in navigating through a website. By employing visualization techniques, the website structure can be presented as regions that provide clarity on all pages, links, and labels. To address these challenges, our goal is to identify the top three existing visualization layouts that improve website navigation. To achieve this, we aim to evaluate different types of existing visualization layouts by applying the region mechanism to visualize website structures. Initially, we designed a region mechanism that ensures scalability and ease of navigation and understanding for website structures. Subsequently, we conducted a technical evaluation to assess the performance of 12 existing visualization layouts on a website structure. During the evaluation, we generated 12 layouts for each webpage in the University of London (UoL) website. We then calculated three metrics for each layout: average aspect ratio, visualization area, and label overlapping. Finally, based on the results obtained, we identified the three most suitable visualization layouts for visualizing a website structure. These selections were made based on low aspect ratio and stability, effective control of the visualization area, and support for readability. Overall, this research focuses on improving website navigation by evaluating existing visualization layouts and selecting the most suitable ones. The chosen layouts are determined based on their ability to maintain low aspect ratio, ensure stability, control visualization area, and support readability.

Aims Digital pathology offers the potential for significant benefits in diagnostic pathology, but currently the efficiency of slide viewing is a barrier to adoption. We hypothesised that presenting digital slides for simultaneous viewing of multiple sections of tissue for comparison, as in those with immunohistochemical panels, would allow pathologists to review cases more quickly. Methods Novel software was developed to view synchronised parallel tissue sections on a digital pathology workstation. Sixteen histopathologists reviewed three liver biopsy cases including an immunohistochemical panel using the digital microscope, and three different liver biopsy cases including an immunohistochemical panel using the light microscope. The order of cases and interface was fully counterbalanced. Time to diagnosis was recorded and mean times are presented as data approximated to a normalised distribution. Results Mean time to diagnosis was 4 min 3 s using the digital microscope and 5 min 24 s using the light microscope, saving 1 min 21 s (95% CI 16 s to 2 min 26 s; p=0.02), using the digital microscope. Overall normalised mean time to diagnosis was 85% on the digital pathology workstation compared with 115% on the microscope, a relative reduction of 26%. Conclusions With appropriate interface design, it is quicker to review immunohistochemical slides using a digital microscope than the conventional light microscope, without incurring any major diagnostic errors. As digital pathology becomes more integrated with routine clinical workflow and pathologists increase their experience of the technology, it is anticipated that other tasks will also become more time-efficient.

: Previous work has identified more than 100 distinct characteristics of data quality, most of which are aspects of completeness, accuracy and consistency. Other work has developed new techniques for visualizing data quality, but there is a lack of research into how users visualize data quality issues with existing, well-known techniques. We investigated how 166 participants identified and illustrated data quality issues that occurred in a 54-file, longitudinal collection of open data. The issues that participants identified spanned 27 different characteristics, nine of which do not appear in existing data quality taxonomies. Participants adopted nine visualization and tabular methods to illustrate the issues, using the methods in five ways (quantify; alert; examples; serendipitous discovery; explain). The variety of serendipitous discoveries was noteworthy, as was how rarely participants used visualization to illustrate completeness and consistency, compared with accuracy. We conclude by presenting a 106-item data quality taxonomy that combines seven previous works with our findings.

Background:Structured Medication Reviews (SMRs) are intended to help deliver the NHS Long Term Plan for medicines optimisation in people living with multiple long-term conditions and polypharmacy. It is challenging to gather the information needed for these reviews due to poor integration of health records across providers and there is little guidance on how to identify those patients most urgently requiring review. Objective:To extract information from scattered clinical records on how health and medications change over time, apply interpretable artificial intelligence (AI) approaches to predict risks of poor outcomes and overlay this information on care records to inform SMRs. We will pilot this approach in primary care prescribing audit and feedback systems, and co-design future medicines optimisation decision support systems. Design:DynAIRx will target potentially problematic polypharmacy in three key multimorbidity groups, namely, people with (a) mental and physical health problems, (b) four or more long-term conditions taking ten or more drugs and (c) older age and frailty. Structured clinical data will be drawn from integrated care records (general practice, hospital, and social care) covering an ∼11m population supplemented with Natural Language Processing (NLP) of unstructured clinical text. AI systems will be trained to identify patterns of conditions, medications, tests, and clinical contacts preceding adverse events in order to identify individuals who might benefit most from an SMR. Discussion:By implementing and evaluating an AI-augmented visualisation of care records in an existing prescribing audit and feedback system we will create a learning system for medicines optimisation, co-designed throughout with end-users and patients.

ABSTRACT Objectives Missing data is the most common data quality issue in electronic health records (EHRs). Checks are typically limited to counting the number of missing values in individual fields, but researchers and organisations need to understand multi-field missing data patterns, and counts or numerical summaries are poorly suited to that. This study shows how set-based visualization enables multi-field missing data patterns to be discovered and investigated. Design Development and evaluation of interactive set visualization techniques to find patterns of missing data and generate actionable insights. Setting and participants Anonymised Admitted Patient Care health records for NHS hospitals and independent sector providers in England. The visualization and data mining software was run over 16 million records and 86 fields in the dataset. Results The dataset contained 960 million missing values. Set visualization bar charts showed how those values were distributed across the fields, including several fields that, unexpectedly, were not complete. Set intersection heatmaps revealed unexpected gaps in diagnosis, operation and date fields. Information gain ratio and entropy calculations allowed us to identify the origin of each unexpected pattern, in terms of the values of other fields. Conclusions Our findings show how set visualization reveals important insights about multi-field missing data patterns in large EHR datasets. The study revealed both rare and widespread data quality issues that were previously unknown to an epidemiologist, and allowed a particular part of a specific hospital to be pinpointed as the origin of rare issues that NHS Digital did not know exist. ARTICLE SUMMARY Strengths and limitations of this study This study demonstrates the utility of interactive set visualization techniques for finding and explaining patterns of missing values in electronic health records, irrespective of whether those patterns are common or rare. The techniques were evaluated in a case study with a large (16-million record; 86 field) Admitted Patient Care dataset from NHS hospitals. There was only one data table in the dataset. However, ways to adapt the techniques for longitudinal data and relational databases are described. The evaluation only involved one dataset, but that was from a national organisation that provides many similar datasets each year to researchers and organisations.

Objectives Missing data is the most common data quality issue in electronic health records (EHRs). Missing data checks implemented in common analytical software are typically limited to counting the number of missing values in individual fields, but researchers and organisations also need to understand multifield missing data patterns to better inform advanced missing data strategies for which counts or numerical summaries are poorly suited. This study shows how set-based visualisation enables multifield missing data patterns to be discovered and investigated. Design Development and evaluation of interactive set visualisation techniques to find patterns of missing data and generate actionable insights. The visualisations comprised easily interpretable bar charts for sets, heatmaps for set intersections and histograms for distributions of both sets and intersections. Setting and participants Anonymised admitted patient care health records for National Health Service (NHS) hospitals and independent sector providers in England. The visualisation and data mining software was run over 16 million records and 86 fields in the dataset. Results The dataset contained 960 million missing values. Set visualisation bar charts showed how those values were distributed across the fields, including several fields that, unexpectedly, were not complete. Set intersection heatmaps revealed unexpected gaps in diagnosis, operation and date fields because diagnosis and operation fields were not filled up sequentially and some operations did not have corresponding dates. Information gain ratio and entropy calculations allowed us to identify the origin of each unexpected pattern, in terms of the values of other fields. Conclusions Our findings show how set visualisation reveals important insights about multifield missing data patterns in large EHR datasets. The study revealed both rare and widespread data quality issues that were previously unknown, and allowed a particular part of a specific hospital to be pinpointed as the origin of rare issues that NHS Digital did not know exist.