In patients with breast cancer, pathological complete response (pCR) has been established as a clinically meaningful surrogate marker for long-term outcomes. While commonly treated with neoadjuvant chemotherapy (NACT), effective treatment decision-making remains challenging, as therapeutic response can vary substantially across patients, calling for predictive models capable of accurately estimating individualized treatment response. To address this, we propose an imaging-based 3D spatio-temporal framework for treatment response prediction that integrates a state-of-the-art graph neural network with relational modeling of temporal interactions across timepoints alongside three novel complementary self-supervised treatment trajectory representation learning objectives. Experiments across a cohort of 585 patients from the public ISPY-2 dataset demonstrate that our method substantially outperforms both vision and self-supervised learning baselines across several classification metrics. Alongside establishing a breast cancer pCR prediction benchmark, we include a principled ablation of our method and further introduce and empirically assess the impact of the available number of DCE-MRI timepoints per patient trajectory and the inclusion of inter-scan time-differences. Overall, our study substantiates the utility of clinically meaningful longitudinal medical imagaging modeling for predicting NACT-induced pCR. We will publicly share our code repository and a user-friendly PyPI library for dataset curation upon publication, effectively promoting reproducible open-source research.

Combining whole-body magnetic resonance imaging (WB-MRI) with clinical variables has the potential to improve systemic disease diagnosis by leveraging complementary sources of patient information. However, structured clinical variables are often incomplete or missing, limiting the applicability of conventional multimodal fusion methods that assume fixed inputs. In this work, we propose TACTIC (Tabular-Attribute Conditioned Transformer for Image Classification), a prompt-based multimodal framework that integrates WB-MRI and structured clinical data through conditional visual feature learning. By encoding clinical attributes as prompts, TACTIC supports an arbitrary number of tabular inputs and naturally handles missing data without requiring imputation or fixed input structures. We evaluate TACTIC on five WB-MRI classification tasks spanning systemic and oncologic applications, including diabetes, chronic obstructive pulmonary disease (COPD), breast cancer, prostate cancer, and metastasis diagnosis. Across all tasks, TACTIC consistently improves performance over image-only baselines when clinical information is available while maintaining strong predictive capability under incomplete tabular inputs. Our results demonstrate the effectiveness of prompt-based models as a flexible approach for improving WB-MRI analysis using clinical context. The model weights and code are available at https://github.com/lauradaza/TACTIC

X-ray dark-field radiography provides complementary diagnostic information to conventional attenuation imaging by visualizing microstructural tissue changes through small-angle scattering. However, the limited availability of such data poses challenges for developing robust deep learning models. In this work, we present the first framework for generating dark-field images directly from standard attenuation chest X-rays using an Uncertainty-Guided Progressive Generative Adversarial Network. The model incorporates both aleatoric and epistemic uncertainty to improve interpretability and reliability. Experiments demonstrate high structural fidelity of the generated images, with consistent improvement of quantitative metrics across stages. Furthermore, out-of-distribution evaluation confirms that the proposed model generalizes well. Our results indicate that uncertainty-guided generative modeling enables realistic dark-field image synthesis and provides a reliable foundation for future clinical applications.

A bstract Human disease risk emerges from the shared influences of genetics, environment, lifestyle, and concurrent diseases over time, resulting in recurring patterns of susceptibility across conditions. However, most risk prediction models treat diseases as independent outcomes or rely on limited input variables, restricting their ability to capture these shared patterns. Here we present RisQ, a framework that learns a unified representation of human health across diseases, modalities, and time. This representation is queried with natural language to estimate disease risk for arbitrary diseases and prediction horizons. Generalization to unseen disease groups and prediction horizons indicates that information is shared across diseases and time, revealing a common structure of disease risk that is learnable. Trained and validated in 488,170 participants from the UK Biobank and evaluated without retraining in 257,538 participants from the independent All of Us cohort, RisQ leverages this shared structure to outperform disease-specific models, multi-disease frameworks, and tabular foundation models in risk prediction. We show that jointly modeling increasing numbers of diseases, input modalities, and prediction horizons improves performance, indicating that scaling these axes increases information transfer and enriches the learned structure. We then show this structure is multi-scale: it captures demographic determinants of disease susceptibility, while also organizing individuals into reproducible cross-disease risk clusters within demographically restricted subgroups. Genetic analyses further support the biological grounding of the structure by linking gene-level loss of function to cross-disease risk profiles. This surfaces known relationships of HBB , SLC22A12 , CASR , and LDLR , while also highlighting less characterized associations. Together, these results indicate that human disease risk exhibits a shared structure that can be learned from multimodal data to improve risk prediction, stratify individuals by cross-disease susceptibility, and support the discovery of relationships across diseases.

Quantitative T2* maps have strong potential for biomarker discovery but are limited by long scan times, rendering them impractical in clinical settings. Significant acceleration can be achieved through undersampling in k-space combined with learning-based reconstruction. However, reconstruction artifacts and noise can propagate into downstream T2* fitting, degrading its accuracy. We introduce CUPA-T2*, a framework that explicitly propagates voxel-wise inter-echo uncertainty from stochastic Monte Carlo dropout reconstructions to downstream T2* fitting via covariance-aware sampling. T2* fitting is performed with a heteroscedastic MLP and a correlation-based regularizer that encourages alignment between predicted variance and reconstruction uncertainty. Experiments on accelerated brain MRI data show tissue-dependent behavior: CUPA-T2* achieves competitive overall T2* fitting performance and improves white-matter performance at higher accelerations. Compared with a heteroscedastic baseline, the proposed framework substantially increases alignment between reconstruction uncertainty and predicted T2* variance, while also revealing a trade-off with calibration (ECE) and selective prediction performance (AURC). CUPA-T2* enables reconstruction uncertainty-aware T2* fitting and delivers voxel-wise uncertainty maps to support the interpretation of quantitative T2* estimates.

Rare diseases represent the long tail of medical imaging, where AI models often fail due to the scarcity of representative training data. In clinical workflows, radiologists frequently consult case reports and literature when confronted with unfamiliar findings. Following this line of reasoning, we introduce RADAR, Retrieval Augmented Diagnostic Reasoning Agents, an agentic system for rare disease detection in brain MRI. Our approach uses AI agents with access to external medical knowledge by embedding both case reports and literature using sentence transformers and indexing them with FAISS to enable efficient similarity search. The agent retrieves clinically relevant evidence to guide diagnostic decision making on unseen diseases, without the need of additional training. Designed as a model-agnostic reasoning module, RADAR can be seamlessly integrated with diverse large language models, consistently improving their rare pathology recognition and interpretability. On the NOVA dataset comprising 280 distinct rare diseases, RADAR achieves up to a 10.2% performance gain, with the strongest improvements observed for open source models such as DeepSeek. Beyond accuracy, the retrieved examples provide interpretable, literature grounded explanations, highlighting retrieval-augmented reasoning as a powerful paradigm for low-prevalence conditions in medical imaging.

Vision-Language Models show strong zero-shot performance for chest X-ray classification, but standard flat metrics fail to distinguish between clinically minor and severe errors. This work investigates how to quantify and mitigate abstraction errors by leveraging medical taxonomies. We benchmark several state-of-the-art VLMs using hierarchical metrics and introduce Catastrophic Abstraction Errors to capture cross-branch mistakes. Our results reveal substantial misalignment of VLMs with clinical taxonomies despite high flat performance. To address this, we propose risk-constrained thresholding and taxonomy-aware fine-tuning with radial embeddings, which reduce severe abstraction errors to below 2 per cent while maintaining competitive performance. These findings highlight the importance of hierarchical evaluation and representation-level alignment for safer and more clinically meaningful deployment of VLMs.

Model merging under unseen test-time distribution shifts often renders naive strategies, such as mean averaging unreliable. This challenge is especially acute in medical imaging, where models are fine-tuned locally at clinics on private data, producing domain-specific models that differ by scanner, protocol, and population. When deployed at an unseen clinical site, test cases arrive in unlabeled, non-i.i.d. batches, and the model must adapt immediately without labels. In this work, we introduce an entropy-adaptive, fully online model-merging method that yields a batch-specific merged model via only forward passes, effectively leveraging target information. We further demonstrate why mean merging is prone to failure and misaligned under heterogeneous domain shifts. Next, we mitigate encoder classifier mismatch by decoupling the encoder and classification head, merging with separate merging coefficients. We extensively evaluate our method with state-of-the-art baselines using two backbones across nine medical and natural-domain generalization image classification datasets, showing consistent gains across standard evaluation and challenging scenarios. These performance gains are achieved while retaining single-model inference at test-time, thereby demonstrating the effectiveness of our method.

The integration of multi-stain histopathology images through deep learning poses a significant challenge. Current approaches struggle with data heterogeneity and missing data, as concatenating multi-stain features may not effectively model stain-specific and cross-stain interactions. We introduce UNICORN (UNiversal stain Integration network for CORonary classificatioN), a two-stage, end-to-end trainable model comprising transformer self-attention blocks to process multi-stain histopathology for atherosclerosis severity prediction. The initial stage employs domain-specific expert models to extract features from each staining. An aggregation expert model then integrates features by learning their interactions. On a multi-class, multi-stain whole slide images (WSIs) dataset of atherosclerotic lesions from Munich Cardiovascular Studies Biobank (MISSION), UNICORN achieved a classification accuracy of 0.68, significantly outperforming state-of-the-art models. UNICORN identifies relevant tissue phenotypes across stainings and implicitly models disease progression. Its explainability and effectiveness in predicting atherosclerosis progression highlight the potential for broader applications in medical research and decision support.

Patch-based methods are widely used in 3D medical image segmentation to address memory constraints in processing high-resolution volumetric data. However, these approaches often neglect the patch's location within the global volume, which can limit segmentation performance when anatomical context is important. In this paper, we investigate the role of location context in patch-based 3D segmentation and propose a novel attention mechanism, LocBAM, that explicitly processes spatial information. Experiments on BTCV, AMOS22, and KiTS23 demonstrate that incorporating location context stabilizes training and improves segmentation performance, particularly under low patch-to-volume coverage where global context is missing. Furthermore, LocBAM consistently outperforms classical coordinate encoding via CoordConv. Code is publicly available at https://github.com/compai-lab/2026-ISBI-hooft

PURPOSE:Radiation-induced pneumonitis (RP) is a side effect after thoracic radiation therapy (RT). The ability to predict RP would facilitate treatment modifications. This study investigates the predictive capacity for symptomatic RP (Common Terminology Criteria for Adverse Events ≥ 2) employing Radiomics and Dosiomics models. METHODS AND MATERIALS:Computed tomography scans, along with physical and 2-Gy equivalent dose volumes (EQD2), dose-volume histograms, and clinical parameters, were evaluated for 708 multicenter lung cancer patients, among whom 89 developed RP ≥ 2. The training cohort consisted of 441 patients from the prospective RTOG 0617 trial. External validation was carried out on 267 patients from the prospective REQUITE (validating pREdictive models and biomarkers of radiotherapy toxicity to reduce side effects and improve QUalITy of lifE in cancer survivors) study. A Random Forest classifier was employed, with feature selection executed within the inner loop of a 10x5-fold nested cross-validation (nCV) utilizing the minimum-redundancy-maximum-relevance algorithm. To address class imbalances, synthetic oversampling and undersampling were implemented using SMOTE-Tomek. The QUANTEC Normal Tissue Complication Probability model served as a reference. Additionally, the experiments were stratified by subgroups (standard/high-dose and 3-dimensional conformal RT (3D-CRT)/intensity-modulated RT (IMRT). RESULTS:The best radiomics model identified in the nCV was trained on the standard-dose subgroup achieved a test ROC-AUC of 0.56. The baseline Normal Tissue Complication Probability model showed a predictive performance with a ROC-AUC of 0.56, which was largely dependent on radiation technique (ROC-AUCS: 3D-CRT: 0.75, IMRT: 0.50). The Dosiomics EQD2 model, trained on the full training cohort, attained the second-best performance in the nCV, demonstrating the same technique-dependence (ROC-AUC of 0.75 vs. 0.39). Using a Dosiomics EQD2 ensemble model trained separately on 3D-CRT and IMRT subgroups increased overall performance to a testing ROC-AUC of 0.61, outperforming other modeling strategies for IMRT, while being outperformed by clinical models for 3D-CRT. CONCLUSIONS:This prospective trial-based study reveals an overall limited predictive capacity of radiomics and dosiomics models and a large influence of radiation technique. IMRT-specific models should be investigated further.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is essential for breast cancer management, but reliance on gadolinium-based contrast agents (GBCAs) restricts use in contraindicated populations, prolongs scan protocols, and presents environmental toxicity concerns. Contrast synthesis offers a non-invasive alternative; however, existing approaches struggle to balance spatial realism with temporal continuity, suffer from slow iterative sampling, underutilize structural priors, and lack clinical validation. We propose a novel conditioned latent transport framework that predicts contrast enhancement in a single forward pass. By anchoring the latent trajectory to the pre-contrast anatomy and applying continuous time conditioning, the model synthesizes patient-specific contrast evolution at any acquisition time. The proposed approach outperforms baseline and the state-of-the-art models across spatial, perceptual, temporal, and distributional metrics. Evaluated on an independent external cohort, the method demonstrates robustness to domain shifts induced by scanner noise as well as differing acquisition protocol. Furthermore, our synthetic contrast enhancement significantly improved downstream tumor segmentation performance, yielding a 22.4

The sharp rise in medical tomography examinations has created a demand for automated systems that can reliably extract informative features for downstream tasks such as tumor characterization. Although 3D volumes contain richer information than individual slices, effective 3D classification remains difficult: volumetric data encode complex spatial dependencies, and the scarcity of large-scale 3D datasets has constrained progress toward 3D foundation models. As a result, many recent approaches rely on 2D vision foundation models trained on natural images, repurposing them as feature extractors for medical scans with surprisingly strong performance. Despite their practical success, current methods that apply 2D foundation models to 3D scans via slice-based decomposition remain fundamentally limited. Standard slicing along axial, sagittal, and coronal planes often fails to capture the true spatial extent of a structure when its orientation does not align with these canonical views. More critically, most approaches aggregate slice features independently, ignoring the underlying 3D geometry and losing spatial coherence across slices. To overcome these limitations, we propose TomoGraphView, a novel framework that integrates omnidirectional volume slicing with spherical graph-based feature aggregation. Instead of restricting the model to axial, sagittal, or coronal planes, our method samples both canonical and non-canonical cross-sections generated from uniformly distributed points on a sphere enclosing the volume. Triangulating these viewpoints yields a spherical graph that captures spatial relationships among views, and we use a graph neural network to aggregate their features accordingly. Experiments across six oncology 3D medical image classification datasets demonstrate that omnidirectional volume slicing improves the average performance in Area Under the Receiver Operating Characteristic Curve (AUROC) from 0.7701 to 0.8154 compared with traditional slicing approaches relying on canonical view planes. Moreover, we can further improve AUROC performance from 0.8198 to 0.8372 by leveraging our proposed graph neural network-based feature aggregation. Notably, TomoGraphView also surpasses large-scale pretrained 3D medical imaging models across all datasets and tasks, underscoring its effectiveness as a powerful framework for volumetric analysis and therefore represents a key step toward bridging the gap until fully native 3D foundation models become available in medical image analysis. We provide a user-friendly library for omnidirectional volume slicing at https://pypi.org/project/OmniSlicer.

Covariance descriptors capture second-order statistics of image features. They have shown strong performance in general computer vision tasks, but remain underexplored in medical imaging. We investigate their effectiveness for both conventional and learning-based medical image classification, with a particular focus on SPDNet, a classification network specifically designed for symmetric positive definite (SPD) matrices. We propose constructing covariance descriptors from features extracted by pre-trained general vision encoders (GVEs) and comparing them with handcrafted descriptors. Two GVEs - DINOv2 and MedSAM - are evaluated across eleven binary and multi-class datasets from the MedMNSIT benchmark. Our results show that covariance descriptors derived from GVE features consistently outperform those derived from handcrafted features. Moreover, SPDNet yields superior performance to state-of-the-art methods when combined with DINOv2 features. Our findings highlight the potential of combining covariance descriptors with powerful pretrained vision encoders for medical image analysis.

In this work, we introduce Progressive Growing of Patch Size (PGPS), an automatic curriculum learning approach for 3D medical image segmentation. Curriculum learning structures the training process by presenting progressively more complex samples to the model, often improving training convergence. In our case, we operationalize this by starting training with small patch sizes and gradually increasing them, which naturally improves the foreground-to-background class voxel ratio in early training stages. We evaluate our approach in two distinct settings. First, a resource-efficient mode maintains a constant batch size throughout training to reduce the input tensor size and computational cost (FLOPs) relative to conventional training. Second, a performance mode inversely scales the batch size relative to the patch volume, keeping the total FLOPs comparable to standard training while maximizing final segmentation quality. Both modes are evaluated on segmentation performance (Dice score) and computational costs across 15 diverse and popular 3D medical image segmentation tasks. The resource-efficient mode matches the segmentation performance of the conventional constant patch size baseline while reducing wall-clock training time to only 44%. We show that the performance mode improves upon the constant patch size baseline, achieving a statistically significant relative gain in mean Dice score of 1.28%. Remarkably, the performance mode surpasses the constant patch size baseline across all 15 tasks, while simultaneously reducing wall-clock training time to only 89%. We found that the benefits are particularly pronounced for tasks with severe foreground-to-background voxel imbalance, such as lesion segmentation. As a consequence of the improved convergence, the proposed performance mode reduces segmentation performance variance relative to conventional constant patch size training, making model comparisons less sensitive to training stochasticity. Finally, our experiments demonstrate that PGPS is not tied to a specific architecture but represents a broadly applicable strategy that consistently boosts performance across diverse segmentation models, including UNet, UNETR, and SwinUNETR. In summary, this simple yet effective transformation of the input sampling strategy substantially improves both segmentation performance and training efficiency, while remaining compatible with diverse segmentation backbones.

In this work, we address the problem of grounding abnormalities in medical images, where the goal is to localize clinical findings based on textual descriptions. While generalist Vision-Language Models (VLMs) excel in natural grounding tasks, they often struggle in the medical domain due to rare, compositional, and domain-specific terms that are poorly aligned with visual patterns. Specialized medical VLMs address this challenge via large-scale domain pretraining, but at the cost of substantial annotation and computational resources. To overcome these limitations, we propose \textbf{Knowledge to Sight (K2Sight)}, a framework that introduces structured semantic supervision by decomposing clinical concepts into interpretable visual attributes, such as shape, density, and anatomical location. These attributes are distilled from domain ontologies and encoded into concise instruction-style prompts, which guide region-text alignment during training. Unlike conventional report-level supervision, our approach explicitly bridges domain knowledge and spatial structure, enabling data-efficient training of compact models. We train compact models with 0.23B and 2B parameters using only 1.5\% of the data required by state-of-the-art medical VLMs. Despite their small size and limited training data, these models achieve performance on par with or better than 7B+ medical VLMs, with up to 9.82\% improvement in $mAP_{50}$. Code and models: \href{https://lijunrio.github.io/K2Sight/}{\textcolor{SOTAPink}{https://lijunrio.github.io/K2Sight/}}.

Test-time adaptation allows pretrained models to adjust to incoming data streams, addressing distribution shifts between source and target domains. However, standard methods rely on single-dimensional linear classification layers, which often fail to handle diverse and complex shifts. We propose Hierarchical Adaptive Networks with Task Vectors (Hi-Vec), which leverages multiple layers of increasing size for dynamic test-time adaptation. By decomposing the encoder's representation space into such hierarchically organized layers, Hi-Vec, in a plug-and-play manner, allows existing methods to adapt to shifts of varying complexity. Our contributions are threefold: First, we propose dynamic layer selection for automatic identification of the optimal layer for adaptation to each test batch. Second, we propose a mechanism that merges weights from the dynamic layer to other layers, ensuring all layers receive target information. Third, we propose linear layer agreement that acts as a gating function, preventing erroneous fine-tuning by adaptation on noisy batches. We rigorously evaluate the performance of Hi-Vec in challenging scenarios and on multiple target datasets, proving its strong capability to advance state-of-the-art methods. Our results show that Hi-Vec improves robustness, addresses uncertainty, and handles limited batch sizes and increased outlier rates.

Cryo-electron tomography provides unique insights into macromolecular complexes in their native environments, yet membrane analysis remains a major bottleneck due to low signal-to-noise ratios, missing wedge artifacts and the complexity of membrane-associated particles. Existing tools often require extensive manual annotation, struggle with generalization across datasets and lack integrated solutions for segmentation, particle localization and quantitative analysis. We introduce MemBrain v2, a deep-learning-enabled framework that unifies these tasks into a streamlined pipeline. MemBrain-seg leverages a diverse, collaboratively generated training dataset and specialized model training strategies to achieve generalizable membrane segmentation across variable tomographic conditions. MemBrain-pick enables data-efficient localization of membrane-bound particles by integrating geometric constraints with deep learning, reducing the need for extensive manual annotation. MemBrain-stats provides quantitative insights into particle distributions, computing spatial metrics to analyze intramembrane particle organization. MemBrain v2 integrates seamlessly into cryo-electron tomography workflows, providing an accessible and structured approach to membrane analysis. MemBrain v2 enables streamlined analysis of membranes in cryo-electron tomography by providing user-friendly tools for membrane segmentation, membrane protein localization and spatial analysis, supporting workflows across diverse datasets.

Accelerated cardiac cine MRI requires reconstructing spatiotemporal images from highly undersampled k-space data. Implicit neural representations (INRs) enable scan-specific reconstruction without large training datasets, but encode content implicitly in network weights without physically interpretable parameters. Gaussian primitives provide an explicit and geometrically interpretable alternative, but their spectra are confined near the k-space origin, limiting high-frequency representation. We propose Gabor primitives for MRI reconstruction, modulating each Gaussian envelope with a complex exponential to place its spectral support at an arbitrary k-space location, enabling efficient representation of both smooth structures and sharp boundaries. To exploit spatiotemporal redundancy in cardiac cine, we decompose per-primitive temporal variation into a low-rank geometry basis capturing cardiac motion and a signal-intensity basis modeling contrast changes. Experiments on cardiac cine data with Cartesian and radial trajectories show that Gabor primitives consistently outperform compressed sensing, Gaussian primitives, and hash-grid INR baselines, while providing a compact, continuous-resolution representation with physically meaningful parameters.

Entropy minimization (EM) is the dominant objective for test-time adaptation, yet its failure mode, model collapse, remains poorly understood. In this work, we show that distribution shifts can cause feature clusters corresponding to distinct classes in the model's representation space to merge, while the decision boundary remains fixed. This induces a systematic skew in the predicted class distribution, referred to as prediction bias. Prediction bias refers to a shift in the predicted class distribution, with some classes overrepresented and others suppressed. We show that entropy minimization amplifies this prediction bias by tightening the existing clusters, reinforcing the incorrect groupings until all predictions collapse to a trivial solution. Next, to demonstrate the significance of prediction bias and mitigate it, we further propose Distribution Shift Bias Reduction (DSBR), a bias-correcting objective that specifically targets this failure mode by equalizing the contribution of each predicted class to the unsupervised entropy minimization loss. To study this failure mode, we design suitable adaptation settings using four medical-imaging datasets and additionally evaluate on ImageNet-C. We find that DSBR consistently stabilizes test-time adaptation, prevents model collapse, and matches or outperforms state-of-the-art methods. Moreover, DSBR operates solely at test-time.