The automatic detection and classification of cardiovascular disease (CVD) from computed tomography (CT) images plays an important role in clinical practice. Recently, a hybrid pipeline (GRC-Net) for CVD classification was proposed, which leverages a deep-learning-based segmentation and registration method to extract radiomic and geometric features. However, GRC-Net relies on a deterministic segmentation mask, without considering the inherent ambiguity associated with cardiac anatomy. In this paper, we propose GRC-ProbNet, which takes advantage of a deep ensemble to produce multiple segmentation masks for a given input. From these masks, we extract multiple uncertainty features. We analyze these uncertainty features for both their correlation with segmentation error and their propagation effects on downstream CVD classification performance. Our experiments on the publicly available MM-WHS and ASOCA datasets show that the uncertainty measure that best reflects segmentation quality is not necessarily the one that provides the strongest signal for downstream CVD classification. Overall, our results demonstrate that GRC-ProbNet utilizing uncertainty features substantially improves CVD classification AUROC (92.92\) compared to the baseline GRC-Net model (91.25%). Our code is publicly available: https://github.com/biomedia-mira/GRC-ProbNet.
AI models are increasingly deployed in clinical practice to assist doctors in diagnostic or screening tasks. However, a critical concern arises from the inherent ability of modern AI models to memorise individual examples from their training datasets. Such memorisation could lead to inaccurate predictions when a model is later used on individuals whose historical data was (potentially unknowingly) used for model training or fine-tuning. In this study, we discover evidence for memorisation bias in two large medical imaging datasets: CheXpert (chest radiography) and Kermany-OCT (optical coherence tomography). Our experiments reveal that a small proportion of data-contributing patients ( 0.6% and 1.1% for CheXpert/Kermany-OCT, respectively) exhibit significant changes in their predictions on (future) longitudinal evaluation data when their historical data is included for model training. Strikingly, we find that larger, more diagnostically accurate models exhibit increased memorisation bias: for Kermany-OCT, the number of data-contributing patients affected by memorisation increases substantially (from 1.1% to 7.2% ) when scaling model size from 1.5 to 80 million parameters. Together, our results raise the question whether the future health outcomes of data-contributing patients could be adversely affected by memorisation bias, i.e., predictions which are biased towards their previous health states.
We study why diffusion autoencoders can achieve similar image quality while learning substantially different latent structures. We trace this behaviour to optimisation dynamics; we analyse curves of image reconstruction against latent representation quality, revealing trajectories that organise around two distinct regimes early in training. Models in the reconstruction regime prioritise image fidelity early, whereas those in the disentanglement regime improve reconstruction and disentanglement more gradually. We hypothesise that this behaviour can be influenced by targeting shortcut pathways in the diffusion U-Net and controlling early noise-level exposure, thereby shaping the reconstruction-disentanglement trade-off during training. To steer optimisation toward stronger representations, we introduce SteeringDRL, combining gated residual U-Nets with a simple noise-level exposure curriculum for training. Across disentanglement benchmarks, SteeringDRL improves representation quality and reduces seed sensitivity. Our method further extends to spatial disentanglement in object-centric learning, improving segmentation quality on synthetic and real-world datasets.
Counterfactual image generation enables controlled data augmentation, bias mitigation, and disease modeling. However, existing methods guided by external classifiers or regressors are limited to subject-level factors (e.g., age) and fail to produce localized structural changes, often resulting in global artifacts. Pixel-level guidance using segmentation masks has been explored, but requires user-defined counterfactual masks, which are tedious and impractical. Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT) addressed this by using segmentation-derived measurements to supervise structure-specific variables, yet it remains restricted to global interventions. We propose Positional Seg-CFT, which subdivides each structure into regional segments and derives independent measurements per region, enabling spatially localized and anatomically coherent counterfactuals. Experiments on coronary CT angiography show that Pos-Seg-CFT generates realistic, region-specific modifications, providing finer spatial control for modeling disease progression.
Aortic shape analysis plays a key role in cardiovascular diagnostics, treatment planning, and understanding disease progression. We present a robust, fully automated pipeline for aortic shape analysis from cardiac MRI, combining deep learning and statistical techniques across segmentation, 3D surface reconstruction, and mesh registration. We benchmark leading segmentation models including nnUNet, TotalSegmentator, and MedSAM2 highlighting the effectiveness of domain specific training and transfer learning on a curated dataset. Following segmentation, we reconstruct high quality 3D meshes and introduce a DL based mesh registration method that directly optimises vertex displacements. This approach significantly outperforms classical rigid and nonrigid methods in geometric accuracy and anatomical consistency. Using the registered meshes, we perform statistical shape analysis on a cohort of 599 healthy subjects. Principal Component Analysis reveals dominant modes of aortic shape variation, capturing both global morphology and local structural differences under rigid and similarity transformations. Our findings demonstrate the advantages of integrating traditional geometry processing with learning based models for anatomically precise and scalable aortic analysis. This work lays the groundwork for future studies into pathological shape deviations and supports the development of personalised diagnostics in cardiovascular medicine.
We introduce the first generative foundation model for chest radiograph synthesis trained from scratch at the billion-parameter scale. Existing radiographic AI models often suffer from poor generalisation across patient subpopulations, institutions, and acquisition settings, resulting in limited real-world clinical utility. Controlled, high-fidelity synthesis of chest radiographs is a promising path toward diversifying clinical datasets and evaluating the robustness of diagnostic models. Therefore, we present the largest specialist generative foundation model for chest radiographs to date, with over 1.3B parameters, trained for 1.6T tokens on a curated, heterogeneous dataset comprising 1.2M radiographs and clinical expert-guided metadata. Our model supports controllable radiograph generation and editing across multiple demographic subgroups, acquisition views, and a dozen pathologies. Moreover, we significantly advance the state of the art in radiograph synthesis fidelity, producing images that are indistinguishable from real radiographs to clinical experts.
Elasmobranchs (sharks and rays) represent a critical component of marine ecosystems. However, they are experiencing global population declines, making effective monitoring essential for their management. Underwater stationary videos, such as those from Baited Remote Underwater Video Stations (BRUVS), are vital for understanding elasmobranch spatial ecology and abundance. However, processing these videos requires time-consuming manual analysis, which can delay conservation efforts. To address this challenge, we developed SharkTrack, a semi-automatic underwater video analysis software. SharkTrack uses Object Detection and Multi-Object Tracking models to automatically detect and track elasmobranchs, providing an annotation pipeline to manually classify elasmobranch species and compute species-specific MaxN (ssMaxN), the standard metric of relative abundance. When tested on BRUVS footage from locations unseen by the model during training, SharkTrack computed ssMaxN with 89% accuracy over 207 h of footage. The semi-automatic SharkTrack pipeline required only two minutes of human labour per hour of video, an estimated 95% reduction in manual analysis time compared to traditional methods. Additionally, we demonstrate the accuracy of SharkTrack across diverse marine ecosystems and elasmobranch species, a significant improvement over previous models, which were limited to specific species or locations. SharkTrack's application extends beyond BRUVS, facilitating the analysis of any underwater stationary video. By making video analysis faster and more accessible, SharkTrack enables research and conservation organizations to monitor elasmobranch populations more efficiently, thereby improving conservation efforts. To further support these goals, we provide public open-access to the SharkTrack software.
Artificial intelligence (AI) health-care technologies offer a means of addressing the growing gap between health-care capacity and demand. However, few technologies have met the complex requirements of health-care systems for adoption. Diabetic eye screening (DES) in England exemplifies the difficulty of understanding these requirements and translating them into real-world implementation decisions. This Review responds to a recognised policy need to develop a target product profile (TPP) for a DES AI system for use in England. The TPP outlines the requirements of the English health-care system for such a device and was developed using a modified Delphi consensus process involving interviews, surveys, and a consensus meeting. Participants included people living with diabetes, health-care professionals, health-care managers and leaders, regulators and policy makers, and developers. Thirty-five product specifications were agreed upon, covering areas such as clinical validity, utility, and environmental sustainability. Our TPP establishes clear criteria for DES AI development and deployment in England, and this TPP development process can serve as a template for initiatives to create TPPs for other AI health technologies and settings.
Image segmentation relies on large annotated datasets, which are expensive and slow to produce. Silver-standard (AI-generated) labels are easier to obtain, but they risk introducing bias. Self-supervised learning, needing only images, has become key for pre-training. Recent work combining contrastive learning with counterfactual generation improves representation learning for classification but does not readily extend to pixel-level tasks. We propose a pipeline combining counterfactual generation with dense contrastive learning via Dual-View (DVD-CL) and Multi-View (MVD-CL) methods, along with supervised variants that utilize available silver-standard annotations. A new visualisation algorithm, the Color-coded High Resolution Overlay map (CHRO-map) is also introduced. Experiments show annotation-free DVD-CL outperforms other dense contrastive learning methods, while supervised variants using silver-standard labels outperform training on the silver-standard labeled data directly, achieving ∼94
Medical artificial intelligence (AI) models hold the promise to improve global access to high-quality diagnostics1. However, the training data underlying these models often contain sensitive patient information that may be exposed through privacy attacks2-7. Previous research has primarily quantified the success of these attacks in aggregate, across all records in a dataset. Thus, the privacy risk faced by individual patients, who often contribute multiple similar records to a training dataset, is poorly understood. Here we present one of the first patient-level privacy audits of AI models for medical diagnostic applications. We focus on membership inference attacks2-4 (MIAs), which seek to determine whether the data of a given individual were used to train a model. Across a diverse range of medical datasets, we show that MIAs can achieve near-perfect success rates for individual patients, even when the aggregate performance does not substantially deviate from random guessing. We further find that the number of patients with high attack success increases substantially with model capacity, and that underrepresented groups-stratified by disease status, self-reported race, insurance, sex or imaging protocol-face disproportionately high attack success. Together, our findings show that aggregate privacy metrics can severely underestimate individual privacy risk. Whether the disparate risk profiles we observe extend to attacks beyond MIAs remains an open question, motivating the further development of risk assessment and mitigation techniques that cater to all data-contributing patients.
Developing predictive models that perform reliably across diverse patient populations and heterogeneous environments is a core aim of medical research. However, generalization is only possible if the learned model is robust to statistical differences between data used for training and data seen at the time and place of deployment. Domain generalization methods provide strategies to address data shifts, but each method comes with its own set of assumptions and trade-offs. To apply these methods in healthcare, we must understand how domain shifts arise, what assumptions we prefer to make, and what our design constraints are. This article proposes a causal framework for the design of predictive models to improve generalization. Causality provides a powerful language to characterize and understand diverse domain shifts, regardless of data modality. This allows us to pinpoint why models fail to generalize, leading to more principled strategies to prepare for and adapt to shifts. We recommend general mitigation strategies, discussing trade-offs and highlighting existing work. Our causality-based perspective offers a critical foundation for developing robust, interpretable, and clinically relevant AI solutions in healthcare, paving the way for reliable real-world deployment.
Foundation models are increasingly used as image feature extractors for mammography, but their robustness under external domain shift remains unclear. We benchmark 15 foundation-model backbones across breast density, BI-RADS severity, and cancer status using a unified frozen-backbone linear-probe protocol, training on 3 source datasets and evaluating on 12 task-compatible out-of-distribution (OOD) datasets after label harmonization. Mammography-specific vision-language models (Mammo-FM and MaMA) provide the strongest mean OOD performance, but robustness is not explained by mammography exposure alone. DINOv3 remains a competitive vision-only baseline, and mammography-adapted pretraining does not consistently improve generalization. Dataset-level analysis further shows that even leading models show heterogeneous performance across datasets. Feature-space inspection reveals that useful representations can preserve clinical signal while retaining dataset and acquisition structure. These findings highlight dataset-level OOD evaluation as a central criterion for assessing mammography representations. Our code is publicly available: https://github.com/biomedia-mira/mammo-ood.
Current methods for radiological classification in traumatic brain injury (TBI), such as the Marshall and Rotterdam score, provide an incomplete description of intracranial lesion burden, rely on time-consuming manual assessments by experts, and are prone to intra- and interrater disagreement. To circumvent these limitations, we previously proposed the Brain Lesion Analysis and Segmentation Tool for Computed Tomography (BLAST-CT), a deep learning-based method using convolutional neural networks (CNNs) to perform multiclass, voxel-wise segmentation and quantification of TBI lesions on CT in an automated fashion. In this study, we expand on our previous work by (1) optimizing the performance of our model using additional training data from CENTER-TBI, (2) externally validating the findings reported in our internal development study by applying our algorithm to an independent imaging dataset from the Prophylaxis for Venous Thromboembolism in Severe Traumatic Brain Injury (PROTEST) multicenter randomized controlled trial. A total of 680 scans from CENTER-TBI were annotated by neuroradiological experts and used to retrain the CNN, creating BLAST version 2.0. Traumatic lesions were subdivided into four distinct classes: intraparenchymal hemorrhage (IPH), extra-axial hemorrhage (EAH), intraventricular hemorrhage (IVH), and perilesional edema. Fifty-one scans from PROTEST were manually annotated to obtain ground-truth lesion labels on an independent imaging dataset. The same PROTEST scans were then contemporaneously run through versions 1.0 and 2.0 of BLAST-CT to evaluate the performance change resulting from the optimization training procedure while calculating segmentation accuracy metrics on an external validation dataset. The additional training phase implemented for version 2.0 yielded an overall mean Dice similarity coefficient (DSC) improvement of 4% when looking at lesions of any size and class (range 0-13% for individual lesion classes). Mean absolute volume errors between automated and ground-truth segmentations also improved for most lesion types (2.94 vs. 1.55 mL for IPH, 18.44 vs. 16.33 mL for EAH, 0.74 vs. 0.80 for IVH, and 1.56 vs. 0.27 for perilesional edema using version 1.0 vs. 2.0, respectively). Overall, the performance of BLAST-CT on the PROTEST external validation dataset was comparable or better to the results obtained on our internal development sample (median DSC for all lesion classes was 0.60 [IQR 0.0-0.94] on the PROTEST images vs. 0.36 [IQR 0.0-0.63] on the CENTER-TBI development dataset). Mean volume differences between the ground-truth and predicted lesion maps were also comparable to the internal sample for most lesion subtypes, with the exception of EAH. We propose one of the first models capable of automated multiclass volumetric lesion segmentation in TBI to be trained and externally validated in a multicenter fashion. After optimizing our model using a large additional training sample from CENTER-TBI, we were able to achieve a level of performance comparable to other state-of-the-art methods. We also make this optimized model (BLAST-CT 2.0) publicly available to provide a robust and generalizable platform for application to large prospective TBI datasets and clinical trials.
Deep neural networks for medical image segmentation are often overconfident, compromising both reliability and clinical utility. In this work, we propose differentiable formulations of marginal L1 Average Calibration Error (mL1-ACE) as an auxiliary loss that can be computed on a per-image basis. We compare both hard-and soft-binning approaches to directly improve pixel-wise calibration. Our experiments on four datasets (ACDC, AMOS, KiTS, BraTS) demonstrate that incorporating mL1-ACE significantly reduces calibration errors, particularly Average Calibration Error (ACE) and Maximum Calibration Error (MCE), while largely maintaining high Dice Similarity Coefficients (DSCs). We find that the soft-binned variant yields the greatest improvements in calibration, over the DSC plus cross-entropy loss baseline, but often compromises segmentation performance, with hard-binned mL1-ACE maintaining segmentation performance, albeit with weaker calibration improvement. To gain further insight into calibration performance and its variability across an imaging dataset, we introduce dataset reliability histograms, an aggregation of per-image reliability diagrams. The resulting analysis highlights improved alignment between predicted confidences and true accuracies. Overall, our approach provides practitioners with explicit control over the calibration-accuracy trade-off, enabling more reliable integration of deep learning methods into clinical workflows. We share our code here: https://github.com/ cai4cai/Average-Calibration-Losses.
Understanding model decisions is crucial in medical imaging, where interpretability directly impacts clinical trust and adoption. Vision Transformers (ViTs) have demonstrated state-of-the-art performance in diagnostic imaging; however, their complex attention mechanisms pose challenges to explainability. This study evaluates the explainability of different Vision Transformer architectures and pre-training strategies—ViT, DeiT, DINO, and Swin Transformer—using Gradient Attention Rollout and Grad-CAM. We conduct both quantitative and qualitative analyses on two medical imaging tasks: peripheral blood cell classification and breast ultrasound image classification. Our findings indicate that DINO combined with Grad-CAM offers the most faithful and localized explanations across datasets. Grad-CAM consistently produces class-discriminative and spatially precise heatmaps, while Gradient Attention Rollout yields more scattered activations. Even in misclassification cases, DINO with Grad-CAM highlights clinically relevant morphological features that appear to have misled the model. By improving model transparency, this research supports the reliable and explainable integration of ViTs into critical medical diagnostic workflows. The code is available at https://github.com/leilibrk/Explainability-of-ViTs .
Deep learning models in medical imaging often fail when deployed in new clinical environments due to distribution shifts in demographics, scanner hardware, or acquisition protocols. A central challenge is underspecification, where models with similar validation performance exhibit divergent real-world failure modes. Although stress testing has emerged as a tool to assess this, current methods typically rely on simple, uninformed perturbations (e.g., brightness or contrast changes), which fail to capture clinically realistic variation and can overestimate robustness. In this work, we introduce a counterfactual stress testing framework based on causal generative models that create realistic "what if" images by intervening on attributes such as scanner type and patient sex while preserving anatomical identity, enabling controlled and semantically meaningful evaluation under targeted distribution shifts. Across two imaging modalities (chest X-ray and mammography), three model architectures, and multiple shift scenarios, we show that counterfactual stress tests provide a substantially more accurate proxy for real out-of-distribution performance than classical perturbations, capturing the direction and relative magnitude of performance changes as well as model ranking. These results suggest that causal generative models can serve as practical simulators for robustness assessment, offering a more reliable basis for evaluating medical AI systems prior to deployment.
Counterfactual generation aims to simulate realistic hypothetical outcomes under causal interventions. Diffusion models have emerged as a powerful tool for this task, combining DDIM inversion with conditional generation and classifier-free guidance (CFG). In this work, we identify a key limitation of CFG for counterfactual generation: it prescribes a global guidance scale for all attributes, leading to significant spurious changes in inferred counterfactuals. To mitigate this, we propose (FCFG), a flexible and model-agnostic guidance technique that enables attribute-wise control following a causal graph. FCFG complements recent advances in classifier-free guidance and can be seamlessly extended to advanced guidance schemes such as CFG++ and APG. Our experiments demonstrate that FCFG significantly improves the axiomatic soundness of inferred counterfactuals across both natural and medical image datasets, mitigating spurious amplification effects, and enhancing counterfactual reversibility.
Foundation models are increasingly adapted for downstream medical imaging tasks, yet the influence of the chosen adaptation strategy on subgroup fairness remains poorly understood. We investigate how three parameter-efficient adaptation techniques, including linear heads on the raw CLS token, an MLP, and an attention-pooling module over multi-layer patch features, affect both pathology classification performance and subgroup disparities when applied to the frozen Rad-DINO chest X-ray encoder. Using MIMIC-CXR, we evaluate eight pathologies across race, sex, and imaging-view subgroups on a prevalence-preserving, demographically balanced test set, and additionally probe how strongly each adapter encodes protected attributes. We find that attention pooling achieves the strongest overall discriminative performance and encodes attributes, particularly race, most strongly, but that improved overall performance does not consistently reduce subgroup disparities. Notably, stronger attribute encoding did not correspond to larger disparities: early network layers encoded race most weakly yet produced the largest subgroup performance gaps. Exploring different attention-pooling layer combinations further revealed no consistent relationship between the layers pooled, attribute encoding strength, and subgroup fairness. Our results indicate that richer, more expressive representations can improve accuracy while leaving fairness implications task-dependent and unpredictable, which must be assessed directly and per-task rather than inferred from encoding strength or overall performance alone.
Automatic detection and classification of Cardiovascular disease (CVD) from Computed Tomography (CT) images play an important part in facilitating better-informed clinical decisions. However, most of the recent deep learning based methods either directly work on raw CT data or utilize it in pair with anatomical cardiac structure segmentation by training an end-to-end classifier. As such, these approaches become much more difficult to interpret from a clinical perspective. To address this challenge, in this work, we break down the CVD classification pipeline into three components: (i) image segmentation, (ii) image registration, and (iii) downstream CVD classification. Specifically, we utilize the Atlas-ISTN framework and recent segmentation foundational models to generate anatomical structure segmentation and a normative healthy atlas. These are further utilized to extract clinically interpretable radiomic features as well as deformation field based geometric features (through atlas registration) for CVD classification. Our experiments on the publicly available ASOCA dataset show that utilizing these features leads to better CVD classification accuracy (87.50 https://github.com/biomedia-mira/grc-net .