The automatic classification of pulmonary nodules is important for early cancer diagnosis. In this work, we aim to find the optimal state space model (SSM) for pulmonary nodule classification by leveraging Differentiable Architecture Search (DARTS). To achieve DARTS, we design the Differentiable State Space Supergraph (DSSS) to search for the optimal SSM. DSSS removes convolution, dimensionality expansion, and spatial attention around the SSM of Mamba while incorporating sampling-based multi-scale feature fusion. Additionally, anatomical scan provides multiple visual sequences of three-dimensional (3D) pulmonary nodules. During the supergraph search process, a knockout (KO) strategy is introduced to progressively remove low-probability candidate operations, thereby reducing the discrepancy between the continuous supergraph and the final discrete architecture while improving search efficiency. The discrete architecture obtained by DSSS is subsequently augmented with output feedback. By stacking the feedback-augmented cells, the final model, termed the Feedback-Augmented State Space Model (FASSM), is constructed. We conducted extensive experiments on the Lung Nodule Analysis 2016 (LUNA16) dataset, and the results show that the FASSM delivers outstanding performance. The architecture search was completed in less than six hours. The resulting FASSM achieved a mean classification accuracy of 93.22% across folds 5-9 with 8.59 M parameters. FASSM achieves a favorable balance between classification performance and model scale, with 8.59 M parameters and a computational cost of 20.41 GFLOPs. Related code and results have been released at: https://github.com/GuYuIMUST/DSSS-FASSM.
Medical lesion segmentation often depends on large, high-quality pixel-level annotations, which are costly to obtain because they require expert delineation. Recent multimodal methods use clinical text to improve visual representation learning, but they usually require paired image-report data during training and text input during inference. This dependency limits their use when paired reports are scarce or when segmentation is performed before report generation. To address this limitation, we propose TeF-SAM, a Segment Anything Model (SAM)-based framework for limited-text training and text-free medical lesion segmentation. TeF-SAM builds a Persistent Region-Informed Semantic Memory (PRISM) from the paired reports available during training. PRISM distils segmentation-relevant textual knowledge into a region-aware prototype space, allowing image features to retrieve semantic cues at inference without online text input. We further introduce Prototype-Driven Regional Lesion Detection (PRLD), which combines prototype-level semantic responses with multi-scale visual features to generate SAM-compatible prompts, including coarse masks, points, and semantic embeddings. We evaluate TeF-SAM on three lesion segmentation benchmarks spanning different anatomies and imaging modalities: QaTa-COV19, MosMedData+, and BUSI. Experimental results show that TeF-SAM achieves competitive segmentation performance using only 1%-15% of the available paired reports and requires no text input during inference. Our code will be released at: https://github.com/Jinxinya2/TeF-SAM.
Predicting the survival time of brain tumor patients remains a critical challenge in the medical prognosis. Existing methods often struggle to integrate heterogeneous data types, such as imaging and clinical records. This paper proposes a novel approach, the Three-Dimensional Feature Fusion Network (3D-FuseNet), designed to address these limitations by integrating high-dimensional imaging features with clinical variables through a late fusion strategy. 3D-FuseNet leverages artificial intelligence (AI) models to automatically extract features from 3D of four MRI sequence images and fuses them with key clinical variables to fully exploit their complementary nature. The network was evaluated using 10 widely-used optimizers and 5 machine learning (ML) methods to assess its performance comprehensively. In the best-case scenario, 3D-FuseNet achieved a mean absolute error (MAE) that is 4% lower than the best baseline model. Furthermore, it reduced the MAE by 2.7% compared to the lowest MAE obtained using individual models such as ResNet50, Unet3D encoder, and Vision Transformer. By effectively integrating multimodal data sources, 3D-FuseNet has demonstrated competitive performance in predicting survival in brain tumor patients, achieving moderate yet consistent improvements over the evaluated baseline. The proposed method outperforms existing architectures and is publicly available at: https://github.com/AIPMLab/3D-FuseNet.
Medical image segmentation often suffers from scarce training data, which limits the performance of deep learning models. Auxiliary task learning offers a promising solution by exploiting complementary supervision signals constructed solely from existing data. However, auxiliary tasks may introduce complex learning signals, and naive integration of multiple tasks often leads to limited performance gains while drastically increasing computational cost. This study presents an adaptive auxiliary task learning method to improve medical image segmentation under scarce training data. The method introduces an adaptive knowledge transfer mechanism that aggregates multi-step model updates from different tasks and model components using learnable weights, enabling more expressive knowledge transfer than prior approaches. To improve training efficiency, a task selection strategy is developed to dynamically select a small subset of beneficial tasks for each training phase. By reusing intermediate computations, it enables the evaluation of task combinations with little overhead. The proposed method is implemented using eight auxiliary tasks with diverse properties. Experiments on five public datasets demonstrate consistent improvements over existing methods, surpassing the best competing approaches by 1.47%, 0.91%, 0.46%, 0.35%, and 2.44% in Dice score. Code is available at: https://github.com/BinyanHu/AdaAuxMedSeg.
Semi-supervised medical image segmentation aims to alleviate the dependence on large-scale annotations by exploiting unlabeled data. However, existing methods often lack effective mechanisms for leveraging semantic knowledge from labeled data and enabling reliable knowledge interaction among unlabeled perturbation branches. To address this, we propose Cross Reliable Knowledge Distillation (CRKD), a semi-supervised framework for medical images. Specifically, we first introduce a Class Semantic Distillation (CSD) module, which transfers class-level semantic knowledge from labeled data to unlabeled representations through class prototype alignment in the feature space, facilitating discriminative feature learning. Then, we propose a Low-Entropy Consistency (LEC) module that dynamically emphasizes reliable low-entropy regions, improving pixel-level prediction consistency under diverse perturbations. Finally, we develop a Cross Knowledge Distillation (CKD) module, consisting of Cross Pixel Knowledge Distillation (CPKD) and Cross Feature Knowledge Distillation (CFKD), which enables reliability-aware bidirectional knowledge transfer in both feature and pixel spaces, allowing perturbation branches to collaboratively correct uncertain representations and predictions without introducing additional network architectures. Extensive experiments demonstrate the effectiveness of CRKD, achieving significant performance gains and robust generalization across diverse imaging modalities and lesion types. Code is available at https://github.com/eeaesa/CRKD.
Prelocalized focal liver lesion classification from multi-phase magnetic resonance imaging remains challenging in multi-center clinical practice because lesion-level annotation is limited, imaging protocols vary across institutions, and routine unlabeled data may contain categories outside the predefined training taxonomy. Existing semi-supervised methods usually assume a closed label space and can therefore be affected by unreliable pseudo-labels when unknown or weakly supported lesions appear in the unlabeled pool. These challenges are closely related to out-of-distribution (OOD) effects caused by category mismatch and clinical distribution shift. To address this problem, we propose OOD-aware Reliability Learning (ORL), a semi-supervised framework for prelocalized liver lesion classification under sparse annotation, partial label-space mismatch, and multi-center distribution shift. ORL learns a compact known-class reference manifold using Prototype-Constrained Representation Learning (PCRL), estimates sample-wise manifold support through Transport-derived Compatibility Estimation (TCE) with Asymmetric Relaxed Optimal Transport (AROT), and combines this support with classifier confidence and prototype affinity to regulate pseudo-label learning. The framework also learns an amortized reliability predictor for inference-time support scoring and reliability-based case ranking after lesion localization. We evaluated ORL on a multi-center liver magnetic resonance imaging cohort and selected retrospective stress settings, including external-center evaluation, unlabeled-pool contamination, missing-phase testing, and composite OOD-oriented score analyses. ORL improved known-category classification over representative semi-supervised baselines and maintained better performance under contamination and distribution shift. These results indicate that manifold support estimation can improve label-efficient prelocalized liver lesion classification and may improve reliability-based case ranking after lesion localization in the evaluated retrospective setting.
Accurate segmentation of coronary arteries from Coronary Computed Tomography Angiography (CCTA) is essential for automated cardiovascular disease diagnosis and risk stratification. However, this task presents a fundamental global-local dilemma: modeling the topology and continuity of the extensive vascular tree requires wide-field contextual reasoning, while delineating thin, elongated distal branches demands high-resolution local processing. In this work, we propose a novel two-stage segmentation framework that explicitly addresses this dilemma through two key innovations. First, we introduce a Global-Local Feature Fusion (GL-Fusion) module. Unlike conventional cascaded approaches that transfer only coarse spatial priors, GL-Fusion treats global features as an associative memory, enabling the local refinement stage to selectively query and integrate semantic context via dual cross-attention mechanisms in both spatial and channel domains. Second, we incorporate an auxiliary SIREN-based Truncated Signed Distance Function (TSDF) regression task to provide geometry-aware supervision for thin vessels. By using sinusoidal activations, the TSDF head is better suited to representing high-spatial-frequency geometric variations than conventional ReLU-based heads, improving vessel overlap and centerline consistency while yielding favorable boundary-distance trends. Evaluated on the public ImageCAS benchmark and a comprehensively annotated in-house FineACA dataset, our method compares favorably with strong baselines, ranking first on all six metrics on ImageCAS and five of six metrics on FineACA. Qualitatively, it yields more complete vessel trees, reduced fragmentation, and Straightened Curved Planar Reformation (SCPR) visualizations with fewer lumen artifacts. Moreover, our framework provides a favorable trade-off between segmentation accuracy and inference efficiency, supporting its potential for practical clinical deployment.
Accurate prostate cancer segmentation is essential for disease progression assessment and prognostic evaluation. Multiparametric MRI (mpMRI) offers complementary structural and functional information, enabling more reliable lesion segmentation than single-modality MRI. However, existing methods still struggle to preserve high-frequency details and handle modality uncertainty, which limits multimodal fusion effectiveness and segmentation accuracy under ambiguous tumor boundaries, complex lesion morphology, and substantial inter-modality heterogeneity. To address these challenges, we propose PCaSFUA-Net, a multimodal prostate cancer segmentation network built upon a pretrained medical vision foundation model, enabling efficient task adaptation with minimal parameter updates. The proposed network integrates a unified multimodal enhancement and fusion (MEF) framework with a coarse-to-fine (C2F) strategy for spatial prompt refinement. Specifically, the MEF framework includes a spatial-frequency collaborative (SFC) module to enhance intra-modality boundary and texture representations and a modality uncertainty-aware fusion (MUAF) module to adaptively reweight multimodal features for robust cross-modal fusion. Extensive experiments on two public datasets and one private dataset show that PCaSFUA-Net consistently outperforms existing methods across multiple evaluation metrics.
Cine cardiovascular magnetic resonance (CMR) provides dynamic sequences for quantifying clinically important indices such as ventricular volumes and ejection fraction. A fast and accurate deep segmentation model is therefore valuable for timely analysis, reducing manual workload and improving reproducibility in routine workflows. However, most existing segmentation models remain largely data-driven and lack explicit physical constraints, which can reduce prediction plausibility under realistic acquisition variability. In this work, we present PGE-UNet, an efficient segmentation framework for cine CMR that incorporates radiofrequency-related information to promote physically plausible outputs. Specifically, it employs the proposed Noise-Aware Encoder with self-supervised noise estimation and smoothing mechanism for robust feature encoding, and the Physics-Regularized Decoder that enforces Maxwell-Helmholtz consistency under a pseudo transmit-field prior. Since transmit-field measurements are typically unavailable in standard protocols, we further introduce a simulator-based procedure to synthesize the pseudo transmit-field prior, enabling physics-regularized training without additional mapping sequences. Experiments on three public cine CMR benchmarks demonstrated strong accuracy in both DICE and HD95, achieving foreground average DICE of 0.9152 on ACDC, 0.8605 on M&Ms, and 0.8548±0.0762 on SCD, while remaining lightweight with only 1.60 million parameters in the default setting. Under CPU-only inference, the default configuration recorded an average latency of 72.38 ms per case, making it suitable for deployment in time-sensitive and resource-constrained environments. Our source code is publicly available at https://github.com/QuocKhanhLuong/PGE-UNet.
Ultrasound image classification remains challenging in clinical practice, given substantial variations in image appearance across devices, acquisition protocols, and operator habits, often unrelated to underlying pathology and associated with inconsistent model performance and limited generalizability. To address this challenge, we propose HAE-BUS (Heterogeneous Acquisition Environments for Breast Ultrasound), a representation learning framework that aims to disentangle pathology-relevant features from acquisition-style bias and improve diagnostic consistency across heterogeneous acquisition conditions. Specifically, HAE-BUS uses Multimodal Large Language Models (MLLMs) as an offline semantic interface to characterize pathology-excluded acquisition cues, from which latent acquisition-style environments are inferred without relying on metadata. These inferred environments are then composed during training to suppress acquisition-style shortcuts while preserving pathology-relevant diagnostic features. Experiments conducted on two breast ultrasound benchmarks, including the public BUSBRA dataset and our private NDTH dataset, indicate that our approach obtains higher AUC and accuracy than the compared methods in the evaluated settings, along with more consistent performance across heterogeneous acquisition conditions, indicating improved robustness and suggesting potential for more reliable clinical translation.
Computed tomography (CT) is essential for clinical diagnosis and radiotherapy planning but involves ionizing radiation, motivating the synthesis of CT images from magnetic resonance imaging (MRI). Most existing MRI-to-CT synthesis methods rely on fully paired supervision, which is difficult to scale in practice and restricts the availability of diverse training data. While unpaired CT samples can provide additional anatomical priors, current approaches typically exploit them as generic style references, lacking explicit mechanisms to preserve anatomical geometry. In this work, we propose DELA-Net, a Dual-Encoder Latent Alignment Network that reformulates MRI-to-CT synthesis as a target-aware latent manifold alignment problem. DELA-Net introduces a geometric reference selection strategy to identify informative unpaired CT samples as latent anchors, enabling structured alignment between paired and unpaired domains. The optimization is decomposed into two coupled alignment tasks - MRI-to-unpaired CT and paired-to-unpaired CT - facilitated by a latent space modulator that adapts MRI representations to the CT manifold while maintaining structural fidelity. Experiments on three benchmark datasets provide promising evidence that DELA-Net achieves a favorable balance across synthesis quality, cross-modality task translation, and bone-related structural preservation in downstream task-driven evaluation. The source code is publicly available at: https://github.com/kennysyp/DELA-Net. The GitHub repository is now publicly accessible.
Accurate segmentation of brain tumors from multimodal magnetic resonance imaging (MRI) is essential for diagnosis, treatment planning, radiotherapy targeting, and longitudinal assessment. Deep learning has advanced this task through convolutional neural networks, Transformers, state-space models, diffusion methods, and foundation models. However, strong benchmark performance does not guarantee clinical reliability because systems remain vulnerable to missing or degraded modalities, cross-center shift, and subregion-specific failures, particularly in enhancing tumor (ET) and tumor core (TC). This survey critically reviews deep learning for multimodal brain tumor segmentation from a deployment-oriented, failure-focused perspective. We introduce a unifying framework linking input reliability, fusion-architecture co-design, subregion-specific failure mechanisms, and uncertainty-aware clinical triage. Using this framework, we compare major architectural paradigms in contextual modeling, boundary preservation, computational feasibility, and robustness across ET, TC, and whole tumor (WT). We reinterpret multimodal fusion as a reliability-allocation problem and examine early, intermediate, token-level, sequence-aware, and adaptive strategies under incomplete or degraded inputs. We also synthesize robust learning approaches for sparse supervision, MRI quality degradation, cross-center variation, and test-time adaptation, and assess interpretability, uncertainty estimation, and human-in-the-loop review as mechanisms for clinical risk control. Finally, our evidence-oriented benchmarking analysis identifies WT performance saturation, persistent ET/TC instability, inconsistent boundary-metric reporting, and insufficient stress testing, center-stratified evaluation, calibration assessment, and computational transparency. We conclude that progress should be judged not only by benchmark accuracy but also by subregion-level reliability under realistic deployment conditions.
Computed Tomography (CT) imaging is frequently compromised by metallic implants, which introduce severe streaking and shading artifacts that obscure anatomical details. While dual-domain networks have advanced Metal Artifact Reduction (MAR), they are limited by their reliance on architecturally homogeneous frameworks. This design ignores the cross-domain asymmetry in artifact manifestation: metal implants appear as broad geometric discontinuities in the sinogram domain but propagate as non-local streaks in the image domain. To bridge this topological discrepancy, we propose an Asymmetric Decoupled Dual-Domain Framework. First, we introduce a Spectral-Geometric Inpainting Module (SGIM) utilizing Fast Fourier Convolution to exploit global spectral consistency for reconstructing missing projections within large gaps. Second, an Anatomical Texture Refinement Module (ATRM) based on Vision Transformers is designed to capture long-range pixel dependencies and mitigate residual streaks. A Residual-Prior Guided Fusion strategy integrates geometrically corrected structural priors from the projection space into the image space. Extensive experiments on the Synthesized DeepLesion dataset demonstrate that our approach outperforms state-of-the-art baselines, achieving a Peak Signal-to-Noise Ratio (PSNR) of 46.49 dB and Structural Similarity (SSIM) of 0.9972. Additionally, the proposed framework exhibits superior generalization on clinical datasets, strong robustness against mask segmentation errors, and high computational efficiency suitable for clinical deployment.
Cardiac magnetic resonance imaging (MRI) plays a critical role in diagnosing cardiovascular diseases; however, acquiring large, annotated datasets remains a significant challenge due to ethical, economic, and logistical constraints. In this paper, we propose a generative framework based on diffusion models for the controllable synthesis of anatomically consistent cardiac MRI scans with corresponding segmentation labels. Our approach combines Low-Rank Adaptation (LoRA) to generate pathology-aware label maps from textual prompts, and ControlNet to guide image synthesis using both semantic and spatial conditioning. This enables the creation of a fully annotated synthetic dataset aligned with cardiac pathologies and phases, supporting supervised training of segmentation models without additional manual labeling. We evaluate the proposed framework in terms of image realism using Fréchet Inception Distance (FID), Kernel Inception Distance (KID), and Fréchet Radiomic Distance (FRD), as well as downstream segmentation performance using a SegFormer model trained on real, synthetic, and combined datasets. Results show that the proposed method improves segmentation accuracy, particularly in data-limited settings. Beyond in-domain evaluation on the ACDC dataset, cross-dataset experiments on the multi-center M&Ms cohort demonstrate improved generalization and robustness to domain shifts. These findings highlight the potential of text-guided diffusion pipelines to generate high-quality, semantically consistent medical imaging data for robust and generalizable AI training.
Accurate and temporally consistent cardiac motion tracking from cine MRI is essential for functional assessment and disease analysis. However, existing learning-based methods are typically formulated in discrete time and struggle to capture long-range temporal dependencies, often leading to accumulated errors and physically inconsistent motion. In this paper, we propose a unified unsupervised framework for cardiac motion tracking based on latent neural ordinary differential equations (ODEs). A frame-aware encoder extracts motion-sensitive features with temporal embeddings, which are evolved in a compact latent space via a neural ODE to model continuous-time deformation dynamics. To capture complex temporal variations in an efficient manner, we incorporate a conditional MLP-based dynamics module and further perform bidirectional forward-backward evolution within a unified framework. The evolved latent representations are decoded into inter-frame deformation fields, and a bidirectional Lagrangian regularization is introduced to enforce long-term temporal consistency and motion reversibility across the cardiac cycle. Extensive experiments on the ACDC and M&Ms datasets demonstrate that the proposed method achieves state-of-the-art performance, producing temporally consistent and physiologically plausible motion fields with a lightweight and efficient architecture.
In magnetic resonance angiography, intracranial vessels exhibit complex tree-like structures with strong anisotropy. Accurate segmentation remains challenging because of discontinuous signals in small vessels and weak contrast in low-signal regions. To overcome these limitations, we propose the 3D Vessel Multi-Directional Attentional Yoked Optimization Network (Vessel-MAYON). The method incorporates two key components. First, the 3D Multi-Directional Core unit captures anisotropic vascular patterns by combining directional convolutions from coronal, sagittal, and axial planes with isotropic cubic convolutions. Second, the Vessel Yoked Attentional Fusion module enhances feature integration through learnable channel weighting, spatial attention, and adaptive residual fusion. Evaluations on the MIDAS dataset and the Cerebral Artery Segmentation Challenge 2023 show Dice Similarity Coefficients of 0.6981 and 0.8476, respectively, outperforming leading approaches in both accuracy and continuity of vascular reconstruction. These results demonstrate that Vessel-MAYON provides reliable, high-precision vessel segmentation and supports clinical application on GPU-equipped workstations. The implementation is publicly available at https://github.com/Brainsmatics/Vessel-MAYON.
Early detection of women's health conditions such as breast cancer, cervical cancer, and polycystic ovary syndrome (PCOS) remains a major challenge because these diseases often require analysing both visual and textual clinical information. Traditional deep learning systems usually depend upon one single source of input - the medical images or the clinical text which restricts them from understanding the full clinical picture. We propose MedFusionNet, a hybrid Transformer-based deep learning model that combines both medical images and clinical reports for more detailed information. The image processing stream introduces EfficientNetB0 and ViT to extract the detailed and global features of images, while in the text processing stream, a transformer based language model identifies the exact meaning of clinical notes. These two models are brought together with a cross-modal influence fusion mechanism that allows the model to learn how the visual patterns use the textual descriptions. After training on 18,434 image-text pairs, MedFusionNet shows good performance after its two-stage training process, learning the fusion and classification layers first, followed by the fine-tuning of all model components. The model achieved very high validation accuracies of 96.8% for breast cancer, 98.2% for cervical cancer, and 99.3% for PCOS detection with strong generalisation ability across diseases. This research further shows that combining image and text inputs will be able to make disease predictions early and accurately, which will make it easy to improve medical support and women's healthcare outcomes.
Pulmonary embolism (PE) is a life-threatening condition for which accurate risk stratification is essential for clinical decision-making. Most existing prognostic models rely on a single data modality, typically structured electronic health records (EHR), limiting their ability to capture the heterogeneous nature of the disease. Multimodal approaches combining EHR, computed tomography (CT) imaging, and free-text clinical reports provide complementary information, but the optimal fusion strategy for prognosis prediction remains unclear. In this study, we present a comprehensive comparative study of multimodal fusion strategies for PE prognosis using the INSPECT dataset, which includes structured EHR, CT images, and clinical text with mortality labels. We develop unimodal baselines and multiple bimodal and trimodal models using early, intermediate, late, and cross-fusion strategies within a unified framework. Models are evaluated for 1-, 6-, and 12-month mortality prediction. Multimodal models consistently outperform unimodal baselines across all prediction horizons, with MCC improvements up to +0.13 for 1-month mortality (0.399 vs 0.269) and +0.11 for 6-month mortality (0.479 vs 0.367). The best overall performance is achieved by late fusion models, reaching an MCC of 0.497 for 12-month prediction, compared to 0.454 for the best unimodal model. Interestingly, bimodal combinations often outperform trimodal fusion in short-term prediction, suggesting that improper integration of multiple modalities may introduce noise and reduce effectiveness. These results highlight the importance of fusion design in multimodal clinical AI and provide practical guidance for developing robust prognosis models for pulmonary embolism. Code is publicly available at https://github.com/arco-group/INSPECT-CS.
Continual Test-Time Adaptation (CTTA) effectively mitigates domain shifts that occur across medical institutions by allowing models to adapt at test time without revisiting the source data. Existing CTTA methods exploit transform invariance through pixel-level augmentation, but these approaches tend to produce inconsistent results in online environments, particularly critical in semantic segmentation tasks. To address these issues, we propose FAWA, a CTTA framework that combines feature-based augmentation and a weighted averaging strategy. FAWA augments features by retrieving statistical information from a continuously updated memory bank, which generates diverse perturbations of input target samples. Each transformed sample is independently fine-tuned, and the resulting parameter sets are merged through continuous weighted averaging. This process leads the model to converge to a stable minimum and mitigates the accumulated error. Experiments on Brain CT and Prostate MRI benchmarks indicate that FAWA achieves competitive performance while ensuring robustness during extended adaptation streams.
