Since the emergence of convolutional neural networks (CNNs) and, later, vision transformers (ViTs), deep learning architectures have predominantly relied on identical block types with varying hyperparameters. We propose a novel block alternation strategy to leverage the complementary strengths of different architectural designs, assembling structurally distinct components similar to Lego blocks. We introduce LegoNet, a deep learning framework that alternates CNN-based and SwinViT-based blocks to enhance feature learning for medical image segmentation. We investigate three variations of LegoNet and apply this concept to a previously unexplored clinical problem: the segmentation of the internal mammary artery (IMA), aorta, and perivascular adipose tissue (PVAT) from computed tomography angiography (CTA) scans. These PVAT regions have been shown to possess prognostic value in assessing cardiovascular risk and primary clinical outcomes. We evaluate LegoNet on large datasets, achieving superior performance to other leading architectures. Furthermore, we assess the model's generalizability on external testing cohorts, where an expert clinician corrects the model's segmentations, achieving DSC > 0.90 across various external, international, and public cohorts. To further validate the model's clinical reliability, we perform intra- and inter-observer variability analysis, demonstrating strong agreement with human annotations. The proposed methodology has significant implications for diagnostic cardiovascular management and early prognosis, offering a robust, automated solution for vascular and perivascular segmentation and risk assessment in clinical practice, paving the way for personalised medicine.
Automatic segmentation of anatomical structures in X-ray images is essential for clinical and research applications, particularly as the increasing volume of medical examinations necessitates workflow automation and efficient data screening. While numerous public X-ray imaging datasets exist, they are predominantly limited to chest X-rays, hindering AI-driven solutions for whole-body segmentation. In this paper, we propose a method for whole-body anatomical segmentation in X-ray images using synthetic data. We generate synthetic X-ray projections from an existing CT dataset using the DiffDRR framework and train five multi-class 2D UNet models, each targeting distinct anatomical groups. To assess generalization, we validate a subset of our models on two real X-ray imaging databases. Our models achieve a perclass median Dice Similarity Coefficient (DSC) above 0.88 for nearly 79 anatomical structures on the synthetic test set and perform on par with models trained on real data for rib segmentation with a similar architecture. We further highlight key challenges in transferring models from simulation to real-world datasets. Our models are made publicly available on GitHub (github.com/risc-mi/totalsegmentator2D) to facilitate further development.
Accurate alignment of histopathological whole slide images (WSIs) across different staining modalities is essential for comprehensive multimodal tissue analysis, yet remains challenging due to complex deformations, sectioning artefacts, and staining variations. This paper presents an efficient registration framework that effectively addresses these challenges through a progressive multi-stage approach. Our approach integrates efficient tissue segmentation using Florence2-SAM2 with a progressive three-stage registration strategy. Initial coarse alignment utilises tissue mask centroids and geometric transformations to establish preliminary correspondence. Subsequently, the accelerated feature extraction module (XFeat) identifies and matches distinctive tissue landmarks from the roughly aligned images, significantly reducing the computational burden of the final stage. The registration concludes with fine alignment using diffusive regulariser based iterative optimisation. Comprehensive evaluation on the ANHIR, ACROBAT and HyReCo datasets demonstrates that our method achieves superior registration accuracy (med-TRE 6.01 m for restained and med-TRE 65.5 m for consecutive sections) and computational efficiency (average runtime of 17 s) compared to state-of-the-art approaches. The proposed framework enables precise spatial correlation of histological features in differently stained modalities, facilitating advanced analysis for both diagnostic and research applications.
In clinical scenarios, radiologists analyse multiple chest Xray (CXR) images from various view positions to identify diseases and abnormalities. To replicate the diagnostic approach of experienced radiologists, we propose an encoder-decoder-based CXR report generation architecture, KCLVA, which leverages the Unified Medical Language System (UMLS) to extract view-specific information from diagnostic reports, focusing on posteroanterior, anteroposterior, and lateral views. This extracted information facilitates view-specific attention (VA) mechanisms and is subsequently used to construct a similarity matrix that enables many-to-many contrastive learning. In the encoder, we employ a knowledge distillation architecture to guide the learning of the student model by freezing the teacher model. Within the student text encoder, the VA mechanism is utilised to automatically assign higher weights to tokens corresponding to a specific view in diagnostic reports based on the view position of the CXR, while assigning lower weights to other tokens. The image and text features are then integrated using contrastive learning. In the decoder, a transformer-based backbone architecture is employed to decode the encoder output and generate a medical diagnosis report. This strategy leverages UMLS to extract view-specific information, employs VA to adjust token weights, and utilises many-to-many contrastive learning through a weighted contrastive loss. Together, these components enable our model to closely simulate the diagnostic process of professional radiologists. Consequently, our method achieves significant improvements of 0.185 on METEOR and 0.078 on ROUGE compared to previous approaches.
Using histological whole slide images (WSIs), how can we stratify patients into meaningful risk groups by leveraging data from both patients with known survival times and event indicators and those without, while requiring no manual post-hoc thresholding of predicted risk scores? Existing survival stratification methods in computational pathology train a supervised model on patients with known survival times and event indicators, then apply it to a test set to generate risk scores. These scores are typically thresholded, often at the median, to assign patients to high- or low-risk groups. Such inductive pipelines overlook the large pool of unlabelled patients even though number of cases with known survival times are typically limited and observed events are even rarer. As a result, existing methods often fail to uncover meaningful risk groups. In this work, we introduce the first Transductive Survival Ranking (TSR) model for WSIs, designed to leverage both labelled and unlabelled data for improved survival prediction. Given a dataset where only a subset of WSIs have associated survival time and event information, our approach (1) ranks patients by predicted survival times, (2) automatically discovers risk groups without requiring manual thresholding, and (3) transduces differential survival patterns from patients with observed events to those without events. We evaluate the proposed approach on multiple pan-cancer datasets, demonstrating that it achieves statistically significant risk sub-grouping in cases where classical approaches fail while producing high concordance indices. Unlike traditional survival models that rely solely on labelled data for training and manual cutoffs, this work leverages transductive learning to make data-driven, threshold-free risk stratification possible, offering a more reliable and data-efficient framework for survival analysis in computational pathology.
