
Interactive foundation models such as SAM struggle to generalize across clinical deployment sites. Among existing adaptation strategies, test-time adaptation (TTA) is the most promising, yet current TTA methods trade one problem for another: removing the physician eliminates clinical grounding, while retaining the physician reintroduces a costly annotation burden. In this work, we propose Clinical-SAM, a two-phase agentic physician-in-the-loop TTA framework that teaches a frozen SAM to approximate what a physician would accept - directly and continuously, without requiring spatial annotations. In Phase I (pre-deployment), a self-prompting agent trained via reinforcement learning generates prompts autonomously, eliminating per-case expert spatial input at inference. In Phase II (deployment), the agent policy is refined online by the one signal physicians already produce: a binary accept/reject feedback on each proposed mask. As the binary feedback accumulates across deployed cases, the agent converges toward a policy that produces masks meeting a reference acceptance standard designed to approximate physician judgment, with progressively fewer interactions. We evaluate Clinical-SAM in a pre-clinical setting on six oncological benchmarks spanning CT (colon, pancreas, liver, kidney) and PET (head & neck, lung) tumor segmentation, using agreement with reference-standard segmentations as a proxy for physician acceptance. Clinical-SAM achieves state-of-the-art performance on all six datasets, outperforming fine-tuning, interactive, TTA, and online-learning baselines, while requiring only 3.3 binary judgments per case on average across the six datasets. Our code is available here.
Cross-modal few-shot medical image segmentation (FSMIS) provides a solution for alleviating annotation scarcity and improving generalization across imaging modalities. However, severe modality discrepancy can disrupt support-query matching, contaminate support-derived foreground and background prototypes, and produce noisy affinity maps, making target-modality segmentation unreliable. To address this problem, we propose CPCLNet, a clustering-inspired prototype consistency learning network for cross-modal FSMIS. CPCLNet formulates cross-modal segmentation as a prototype consistency learning problem and progressively improves feature-space consistency, prototype reliability, and affinity-map coherence. Specifically, a parameter-sharing feature encoder provides a common representation basis across modalities, and a cross-modal distribution alignment module performs episode-specific support-query alignment. A clustering-inspired representative prototype descriptor further suppresses unstable support responses and constructs compact, modality-robust prototypes. Finally, an RWKV-based affinity map prediction module propagates prototype-guided correspondence across long-range spatial contexts with linear complexity. Extensive experiments on three imaging modalities, demonstrate that CPCLNet outperforms existing methods, achieving an average Dice improvement of 3.34 percentage points over the second-best competitor across CT→MRI and CT→PET tasks under both 1-shot and 5-shot settings. Further integration with MedSAM yields consistent improvements, indicating CP-CLNet’s complementary potential with foundation models.
The Segment Anything Model (SAM) has shown strong zero-shot segmentation capabilities across general visual domains. However, its effectiveness in Medical Image Segmentation (MIS) remains constrained by the medical-domain gap, reliance on explicit user prompts, and limited adaptability to small, heterogeneous, or illdefined targets. In particular, feature adaptation in existing SAM approaches often remains shared across anatomical targets, making it difficult to tailor local representations to target identity, morphology, and surrounding anatomical context. In this paper, we propose Dynamic Region-Adaptive Modulated SAM (DReAM-SAM), a fully automatic framework that introduces target- and region-adaptive representation learning for fine-grained MIS. Our framework introduces three key technical pillars: (1) Target-Conditioned Dynamic Modulation: We propose an agile CNN (ACNN) that complements SAM’s encoder by adapting local feature extraction to target identity and morphology while promoting boundary-sensitive details. (2) Query-Adaptive Feature Interaction: We develop a bidirectional neighborhood attention mechanism to integrate complementary ACNN and SAM features over query-adaptivemulti-scale contexts. (3) Target-Aware Self-Prompting: We further devise a self-prompting mechanism that exploits target-existence cues and target-aware representations to automatically generate surrogate prompts. Extensive experiments across six MIS datasets and diverse medical imaging modalities reveal that DReAM-SAM outperforms state-of-the-art SAM variants and task-specific MIS models. The code is publicly available at https://github.com/ku-milab/DReAM-SAM.
The Segment Anything Model (SAM) has demonstrated groundbreaking performance in natural image segmentation, yet its direct application to medical imaging remains suboptimal due to domain shifts in data distributions and the inherent 3D nature of medical data. Although recent SAM-based methods have employed parameter-efficient transfer learning (PETL) to adapt SAM for medical tasks, they often overlook the critical 3D contextual information essential for accurate volumetric segmentation. To address this limitation, we introduce VSS-SAM++, a novel dual-branch architecture that combines SAM's foundational visual priors with Vision Mamba's capacity for modeling long-range spatial dependencies. In this framework, SAM serves as the primary encoder for high-level feature extraction, while a parallel Mamba branch captures cross-slice dependencies in 3D medical volumes. A gated hybrid attention module then dynamically fuses complementary features from both branches, adaptively weighting multi-view representations to minimize feature ambiguity and enhance segmentation precision. Extensive evaluations across nine public CT and MRI datasets demonstrate that VSS-SAM++ outperforms existing methods by 0.2-11.3% in Dice score on diverse organ segmentation tasks. The framework's robustness to domain shifts and scalability across diverse modalities highlights its potential for clinical deployment.