Interactive 3D medical image segmentation methods typically require manual bounding box prompts, limiting their applicability in automated workflows. In this work, we propose an intensity-based thresholding strategy that automatically generates bounding box prompts when explicit annotations are unavailable. Our method leverages statistical properties of medical images to identify regions of interest through adaptive thresholding, morphological operations, and connected component analysis. Experiments on the CVPR BiomedSegFM dataset demonstrate that this automated prompting strategy significantly improves segmentation performance in high-contrast modalities, achieving 0.73 DSC for CT and 0.74 DSC for PET, compared to 0.68 and 0.59 respectively with standard prompts. However, the method faces challenges in low-contrast modalities such as Ultrasound, where performance decreases from 0.68 to 0.31 DSC due to speckle noise and ambiguous tissue boundaries. We also report preliminary experiments with lightweight MobileNet encoders as alternatives to Vision Transformers, finding that current lightweight architectures suffer substantial accuracy degradation in 3D medical segmentation tasks. Our results highlight both the promise and limitations of automated prompt generation for multi-modality medical imaging. Code: https://github.com/lexorcvpr/lexor-cvpr-2025/ .
3D biomedical image segmentation is a critical technology for clinical diagnostics, surgical planning, and disease analysis. While foundation models such as SAM and its medical derivatives have achieved remarkable progress, their adaptation to volumetric images remains limited, particularly in terms of diverse imaging modalities and the need for efficient user interaction. To advance research in this field, CVPR 2025 Interactive 3D Biomedical Image Segmentation Challenge was established. We propose DCM (DualClickMed) as a solution to this challenge, with a dual-expert architecture featuring both global and local Region-of-Interest (RoI) strategies. The global-RoI expert provides comprehensive anatomical context by processing the entire organ based on user prompts, while the local-RoI expert focuses on high-resolution patches centered on specific user clicks, enabling precise segmentation of fine structures. We further introduce tailored prompt simulation strategies for each expert, closely mimicking real-world interactive behaviors during training. Extensive experiments on challenge dataset covering five modalities demonstrate that our approach outperforms baselines, with final DSC scores of 0.8533 (CT), 0.6880 (MRI), 0.6003 (Microscopy), 0.7864 (PET), and 0.9385 (Ultrasound), achieving significant improvements in both region overlap and boundary accuracy metrics.
The increasing volume of complex 3D biomedical imaging data highlights the need for accurate and efficient analysis methods. Segmentation of such data is essential for diagnosis, anatomical analysis, disease monitoring, and treatment planning. However, existing segmentation algorithms often struggle with the variability of object structures and the diversity of imaging modalities. To address these challenges, we introduce iMedSTAM, a promptable foundation model for 3D image and video segmentation. The model is also capable of progressively improving segmentation quality based on user interactions. iMedSTAM was developed by fine-tuning EfficientTAM on a large-scale dataset comprising over 270,000 3D image–mask pairs and 4,000 video–mask pairs, covering five different medical imaging modalities. In addition, we extend the EfficientTAM architecture with a bidirectional inference and memory mechanism that enables the processing of volumetric data. iMedSTAM significantly outperforms all previous models on the publicly available validation set in the coreset track and achieves state-of-the-art results in the all-data track. On the test set, our model reaches an average final DSC and NSD of 0.805 and 0.842, respectively. For DSC_AUC and NSD_AUC, which measure the cumulative improvement through additional user interactions, iMedSTAM achieves scores of 3.129 and 3.258.
Existing 3D medical image segmentation methods are often constrained by a fixed set of predefined classes or by reliance on manually defined prompts such as bounding boxes and scribbles, which are often labor-intensive and prone to ambiguity. To address these limitations, we present a framework for 3D medical image segmentation across diverse modalities guided solely by free-text descriptions of target anatomies or diseases. Our solution is built on a multi-component architecture that integrates efficient feature encoding via decomposed 3D convolutions and self-attention, multi-scale text-visual alignment, and a SAM-inspired mask decoder with iterative refinement. The model is further conditioned through a prompt encoder that transforms language and intermediate visual cues into spatially aligned embeddings. To train and evaluate our model, we used a large-scale dataset of over 200,000 3D image-mask pairs spanning CT, MRI, PET, ultrasound, and microscopy. Our method achieved an average Dice of 0.6091 and F1_50 score of 0.1131 on the open validation set, outperforming baselines such as CAT (Dice 0.5316, F1_50 0.1935) and SAT (Dice 0.5573, F1_50 0.0956). It showed strong generalization across modalities, with particularly high performance on ultrasound (Dice 0.8337) and CT (Dice 0.6707). These results confirm the feasibility of free-text-guided 3D segmentation and establish our approach as a strong foundation model for general-purpose medical image segmentation. Our code is publicly available at: https://github.com/mirthAI/Text3DSAM/ .
Medical image segmentation is a critical task in clinical diagnostics and biomedical research. While deep learning has significantly advanced the field, most existing methods rely on task-specific models that require extensive manual annotations for training or adaptation. Vision foundation models, such as the Segment Anything Model (SAM), offer a promising alternative with their universal segmentation capabilities. However, their application to 3D medical imaging remains limited, especially in zero-shot scenarios involving previously unseen anatomical structures. In this work, we introduce GAMT, a zero-shot, training-free framework that repurposes powerful 2D foundation segmentation models (e.g., SAM, SAM-Med2D) for universal 3D biomedical image segmentation. To bridge the dimensionality gap, GAMT performs slice-wise inference along three orthogonal anatomical planes (axial, coronal, and sagittal) and subsequently fuses the predictions to construct a coherent 3D segmentation mask. Crucially, without any model training or fine-tuning, this framework achieves average Dice Similarity Coefficient (DSC) and Normalized Surface Dice (NSD) scores of 0.487 and 0.477, respectively—without requiring model training or fine-tuning. Our code and results are publicly available at https://github.com/SpatialAILab/GAMT.
Medical image segmentation has advanced considerably with foundational models like the Segment Anything Model (SAM) and its medical variants, yet real-world clinical deployment remains constrained by heterogeneous imaging protocols, limited data generalization, and the inefficiency of manual interaction. While recent SAM-based frameworks (e.g., SAM2, MedSAM2) introduce memory-aware mechanisms, they still rely on dense re-encoding and lack targeted correction strategies. We propose “From Single-Round to Sequential: Building Stateful Interactive Segmentation with SegVol and GRU Corrector”, a lightweight framework that reformulates interactive segmentation as a sequential refinement process guided by uncertainty and error heuristics. Specifically, we design: (1) a GRU-based temporal module to encode interaction history and enable stateful correction, and (2) an uncertainty-driven region adaptation scheme that selectively focuses refinement on ambiguous or mis-segmented areas, reducing redundant computation while improving correction efficiency. On validation data, our framework achieves a progressive Dice coefficient improvement from 0.661 (single-box prompt) to 0.671 after three refinement rounds, showing a 1.5
Interactive segmentation of 3D medical images seeks to produce accurate object masks with minimal user input, substantially alleviating the burden of manual annotation. For the CVPR 2025 Foundation Models for Interactive 3D Biomedical Image Segmentation Challenge, we extend the VISTA3D foundation model—a state-of-the-art 3D segmentation network supporting both automatic and interactive modes—by introducing several targeted improvements for robust interactive segmentation. First, we propose a Gaussian Edge-Center point sampling strategy, which leverages Gaussian-weighted randomness combined with center/edge distance transforms to preferentially sample points at object centers and boundaries. This yields more realistic and effective foreground/background click simulations during training. Second, we integrate this sampler into a two-stage fine-tuning pipeline: initial conventional fine-tuning with provided pre-trained weights, followed by prompt-focused fine-tuning using our improved sampling strategy. Third, to meet the challenge’s 90-second runtime limit, we optimize inference by dynamically adjusting the region of interest (ROI) size and resolution based on input voxel spacing, including adaptive downsampling and ROI cropping. We trained models for both tracks—using 4×A100 GPUs for the full dataset and 4×A800 GPUs for the 10 https://github.com/M4cheal/GS_MedSegFM .
Three-dimensional (3D) image segmentation plays a pivotal role in clinical diagnosis, therapy planning, and drug discovery by enabling the precise delineation of anatomical structures, pathological lesions, and cellular features in medical imaging modalities such as CT and MRI, as well as in biomedical microscopy. Despite its central importance, 3D segmentation remains a formidable technical challenge due to high computational requirements, the vast diversity of segmentation tasks across clinical and research domains, and the lack of interoperability among existing models, which are typically developed for specific modalities and tasks. To address these limitations, we introduce BiomedParse-V, a scalable and generalizable multimodal foundation model that leverages pretrained 2D foundation models to enable accurate, text-prompted 3D image segmentation. Our method features a novel Fractal Volumetric Encoding (FVE) scheme, which hierarchically compresses volumetric data by capturing self-similarity across slices into a compact fractal-based 2.5D representation. This design allows the effective use of powerful 2D foundation models while preserving essential 3D spatial context. We further propose the Independent Segmentation Discriminator (ISD) module to promote robust and consistent object localization throughout the segmented volume, addressing the challenges of maintaining spatial coherence in text-guided segmentation. Extensive experiments conducted across diverse biomedical imaging modalities demonstrate that BiomedParse-V consistently achieves state-of-the-art performance, significantly surpassing leading supervised 3D segmentation models. Our approach delivers a prompt-driven, computationally efficient, and broadly applicable solution for 3D biomedical image segmentation, advancing the accessibility and impact of segmentation technologies in real-world clinical and research environments.
We present a multimodal approach to open-vocabulary segmentation in medical imaging by training five modality-specific models using a unified architecture based on the SAT model. Each model is tailored to a specific imaging modality—CT, MRI, Ultrasound, Microscopy, and PET, while maintaining architectural consistency to ensure comparability and generalizability. To address the challenge of limited data availability, particularly in modalities like Ultrasound and Microscopy, we implement distinct sampling strategies designed to maximize anatomical and pathological diversity across training cases. We aim to evaluate the effectiveness of open-vocabulary segmentation across diverse medical imaging modalities using consistent text prompts and unified label representations. For CT, MRI, and Ultrasound, performance is reported using Dice Similarity Coefficient (DSC) and Normalized Surface Dice (NSD), while for Microscopy and PET, we follow challenge-specific guidelines and report F1 scores. On the official test set, the models achieved: CT (DSC: 0.2884, NSD: 0.2114), MRI (DSC: 0.1644, NSD: 0.1474), Microscopy (F1: 0.4502), and PET (F1: 0.0728). These results demonstrate the viability of modality-specific training within an open-vocabulary framework and provide a foundation for further improvements.
Accurate segmentation of anatomical and pathological structures in 3D medical imaging is critical for effective diagnosis, treatment planning, and disease monitoring. Despite recent advances in deep learning, automated 3D medical image segmentation remains challenging due to anatomical variability, imaging artifacts, and the limited availability of annotated data. To address these issues, we present an interactive segmentation framework in the SAM-Med3D architecture with an xLSTM-UNet image encoder. Our encoder is specifically designed to capture long-range dependencies and hierarchical spatial features in volumetric medical data, improving contextual awareness while maintaining computational efficiency. We validate our approach using the CoreSet from the CVPR 2025 Foundation Models for 3D Biomedical Image Segmentation Challenge. Initial results demonstrate that our model achieves competitive performance in limited-scale testing, with DSC Final scores of 0.4855 (CT), 0.3071 (MRI), 0.4070 (PET), and 0.4458 (Ultrasound. NSD Final scores follow a similar trend, reaching 0.4992 (Ultrasound) and 0.4545 (CT). These early findings suggest strong potential for our architecture, particularly with further training on the full dataset. The proposed model supports multimodal prompts, including points and bounding boxes, allowing for flexible and intuitive user interaction a key requirement in clinical workflows. Our contributions include the development of a 3D-optimized interactive segmentation encoder, its integration into an existing foundation model framework, and an empirical evaluation that highlights the feasibility of our design. Future work will focus on full-scale training and refinement to bridge the performance gap with state-of-the-art methods.
We introduce Medal S, a medical segmentation foundation model that supports native-resolution spatial and textual prompts within an end-to-end trainable framework. Unlike text-only methods lacking spatial awareness, Medal S achieves channel-wise alignment between volumetric prompts and text embeddings, mitigating inaccuracies from resolution mismatches. By preserving full 3D context, it efficiently processes multiple native-resolution masks in parallel, enhancing multi-class segmentation performance. A lightweight 3D convolutional module enables precise voxel-space refinement guided by both prompt types, supporting up to 243 classes across CT, MRI, PET, ultrasound, and microscopy modalities in the BiomedSegFM dataset. Medal S offers two prompting modes: a text-only mode, where model predictions serve as spatial prompts for self-refinement without human input, and a hybrid mode, incorporating manual annotations for enhanced flexibility. For 24-class segmentation, parallel spatial prompting reduces inference time by more than 90% compared to sequential prompting. We propose dynamic resampling to address target-patch ratio imbalance, extending SAT and nnU-Net for data augmentation. Furthermore, we develop optimized text preprocessing, a two-stage inference strategy, and post-processing techniques to improve memory efficiency, precision, and inference speed. On the five-modality average on the validation set, Medal S outperforms SAT with a DSC of 75.44 (vs. 69.83), NSD of 77.34 (vs. 71.06), F1 of 38.24 (vs. 24.88), and DSC TP of 65.46 (vs. 46.97). Medal S achieves excellent performance by harmonizing spatial precision with semantic textual guidance, demonstrating superior efficiency and accuracy in multi-class medical segmentation tasks compared to sequential prompt-based approaches. Medal S will be publicly available at https://github.com/yinghemedical/Medal-S .
Interactive 3D biomedical image segmentation requires efficient models that can iteratively refine predictions based on user prompts. Current foundation models either lack volumetric awareness or suffer from limited interactive capabilities. We propose a training strategy that combines dynamic volumetric prompt generation with content-aware adaptive cropping to optimize the use of the image encoder. Our method simulates realistic user interaction patterns during training while addressing the computational challenges of learning from sequential refinement feedback on a single GPU. For efficient training, we initialize our network using the publicly available weights from the nnInteractive segmentation model. Evaluation on the Foundation Models for Interactive 3D Biomedical Image Segmentation competition demonstrates strong performance with an average final Dice score of 0.6385, normalized surface distance of 0.6614, and area-under-the-curve metrics of 2.4799 (Dice) and 2.5671 (NSD).
We present ENSAM (Equivariant, Normalized, Segment Anything Model), a lightweight and promptable model for universal 3D medical image segmentation. ENSAM combines a SegResNet-based encoder with a prompt encoder and mask decoder in a U-Net-style architecture, using latent cross-attention, relative positional encoding, normalized attention, and the Muon optimizer for training. ENSAM is designed to achieve good performance under limited data and computational budgets, and is trained from scratch on under 5,000 volumes from multiple modalities (CT, MRI, PET, ultrasound, microscopy) on a single 32 GB GPU in 6 h. As part of the CVPR 2025 Foundation Models for Interactive 3D Biomedical Image Segmentation Challenge, ENSAM was evaluated on the hidden test set with multimodal 3D medical images, obtaining a DSC AUC of 2.404, NSD AUC of 2.266, final DSC of 0.627, and final NSD of 0.597, outperforming two previously published baseline models (VISTA3D, SAM-Med3D) and matching the third (SegVol), surpassing its performance in final DSC but trailing behind in the other three metrics. In the coreset track of the challenge, ENSAM ranks 5th of 10 overall and best among the approaches not utilizing pretrained weights. Ablation studies confirm that our use of relative positional encodings and the Muon optimizer each substantially speed up convergence and improve segmentation quality.
In this paper, we propose a novel approach for perceptual quality assessment (PQA) of AI-generated images by incorporating style awareness and uncertainty-guided probabilistic modeling. The proposed method, Uncertaintyguided Style-aware Probabilistic Perceptual Quality Assessment (US-PPQA), takes advantage of the distinct characteristics of different image styles, such as anime and realistic, to improve quality predictions. We use the AGIQA-1K dataset, which contains 1080 AI-generated images across two styles, to train separate models that predict quality scores for each style. Additionally, we introduce a style classification model that estimates the probability of an image belonging to either style. The final quality score is then computed by combining the style-specific predictions, weighted by these probabilities. Furthermore, we incorporate uncertainty into the individual style quality score predictors and recompute the final score. The performance of our method is evaluated using standard correlation metrics (SRCC, KRCC, and PLCC). Our results demonstrate that the style-aware, uncertainty-guided model outperforms traditional methods, achieving improved rank accuracy in perceptual quality assessment for AI-generated content.
In this paper, we review the NTIRE 2025 challenge on single-image reflection removal (SIRR) in the wild. SIRR is a fundamental task in image restoration. Despite progress in academic research, most methods are tested on synthetic images or limited real-world images, creating a gap in realworld applications. In this challenge, participants are required to process real-world images that cover a range of reflection scenarios and intensities, with the goal of generating clean images without reflections. The challenge attracted more than 200 registrations, with 11 of them participating in the final testing phase. The top-ranked methods advanced the state-of-the-art reflection removal performance and earned unanimous recognition from the five experts in the field. The proposed datasets are available at https://huggingface.co/datasets/qiuzhangTiTi/NTIRE2025-SIRR and the homepage of this challenge is at https://github.com/caijie0620/Reflection-Removal-in-thewild.
The Segment Anything Model (SAM) is widely used for segmenting a diverse range of objects in natural images from simple user prompts like points or bounding boxes. However, SAM's performance decreases substantially when applied to non-natural domains like microscopic imaging. Furthermore, due to SAM's interactive design, it requires a precise prompt for each image and object, which is un-feasible in many automated biomedical applications. Previous solutions adapt SAM by training millions of parameters via fine-tuning large parts of the model or of adapter layers. In contrast, we show that as little as 2,048 additional parameters are sufficient for turning SAM into a usecase specialist for a certain downstream task. Our novel PTSAM (prompt-tuned SAM) method uses prompt-tuning, a parameter-efficient fine-tuning technique, to adapt SAM for a specific task. We validate the performance of our approach on multiple microscopic and one medical dataset. Our results show that prompt-tuning only SAM's mask decoder already leads to a performance on-par with state-of-the-art techniques while requiring roughly 2,000× less trainable parameters. For addressing domain gaps, we find that additionally prompt-tuning SAM's image encoder is beneficial, further improving segmentation accuracy by up to 18% over state-of-the-art results. Since PTSAM can be reliably trained with as little as 16 annotated images, we find it particularly helpful for applications with limited training data and domain shifts.
Audio-visual classification models typically assume the availability of all modalities at inference time. However, real-world conditions often result in missing or corrupted modalities due to noise, sensor failures, or transmission errors. To address this challenge, we propose Prompt the Missing, a lightweight and robust framework that leverages prompt learning to adaptively handle uncertain modality availability. Our method introduces learnable prompt tokens at both the input and attention levels, enabling dynamic adjustment to various degradation scenarios without modifying the backbone. We further employ a case-wise training strategy that simulates diverse missing-modality conditions, allowing the model to generalize effectively. Experiments on UrbanSound8K-AV and CIFAR10-AV show that our approach matches full fine-tuning performance under complete inputs, and significantly outperforms existing baselines under missing-modality settings-achieving up to +10.4% accuracy gain while reducing training time by 96% and memory usage by 82.3%. Our model also consistently surpasses parameter-efficient tuning methods such as LoRA and Adapter, with ablation studies confirming the effectiveness of our prompt design, fusion mechanisms, and prompt length choices. Notably, even under Concat-based evaluation—where degradation types are unknown—our method outperforms full fine-tuning, demonstrating strong generalization and deployment readiness. Code is available at https://github.com/pej0918/Prompt-The-Missing.
Deep neural networks (DNNs) have achieved remarkable success across various domains, yet their rigid, static computation graphs lead to significant inefficiencies in real-world deployment. Standard architectures allocate equal computational resources to all inputs, disregarding their inherent complexity, which results in unnecessary computation for simple samples and suboptimal processing for complex ones. To address this, we propose the Complexity-Aware Routing Network (CARN), a novel framework that dynamically adjusts computational pathways based on input complexity. CARN integrates a self-supervised complexity estimation module that quantifies input difficulty using confidence, entropy, and computational cost, guiding a neural network-based routing mechanism to optimally assign task modules. The model is trained using a routing loss function that balances assignment accuracy and computational efficiency, mitigating expert starvation while preserving specialization. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that CARN achieves up to 4× reduction in computational cost and over 10× reduction in parameter movement while maintaining high accuracy compared to state-of-the-art static models. The code and pre-trained models are made available at https://github.com/rrgaire/CARN for reproducibility and further research.
Deep neural networks face a fundamental trade-off between computational efficiency and accuracy. This paper introduces a method for network depth optimization that enables flexible inference with adjustable computational budgets while potentially improving training dynamics. Our approach partitions each residual stage into core and gated sub-paths, employing depth-aware training to develop networks that can operate at varying depths. We present theoretical analysis of our method through three key results: (1) an explicit regularization theorem quantifying how our training approach may penalize discrepancies between network configurations, (2) a statistical convergence theorem suggesting tighter generalization bounds based on effective network depth, and (3) a gradient dynamics theorem characterizing the noise properties induced by our training procedure. Empirically, our method shows improvements over conventional approaches on standard benchmarks, achieving favorable accuracy-efficiency trade-offs with a single trained model. The Gated Depth architecture provides a framework for deploying deep networks across diverse computational environments.
RGB photo reconstruction from RAW camera images is an increasingly popular deep learning problem with a practical application to mobile cameras. This creates a need for solutions that are not only performant but are additionally compatible with real mobile AI hardware such as GPUs or NPUs. In this Mobile AI challenge, we address this problem and propose the participants to design efficient learned ISP models that can demonstrate fast inference times on mobile GPUs. For this, the participants were provided with a largescale Fujifilm UltraISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102MP medium format FujiFilm GFX100 camera. The runtime of all models was evaluated on the latest Adreno and Mali GPUs used in Qualcomm and MediaTek chipsets. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos under 30 ms in the majority of cases and delivering high-fidelity results. A comprehensive description of the models developed in the challenge is provided in this paper.
