Video MLLMs often struggle with fine-grained spatio-temporal reasoning, sometimes generating correct answers based on irrelevant frames or objects. Although outputting spatio-temporal evidence during reasoning is a promising direction, existing RL frameworks typically rely on geometry-only (IoU) rewards, which can be sensitive to boundary perturbations and overlook semantic alignment. To address this, we propose Semantic Evidence Reward (SER), which reformulates spatio-temporal evidence grounding as a constrained verification task. Instead of computing pixel-level overlap, SER uses a referee VLM as a local checker to evaluate model-generated evidence claims across two dimensions: relevance and localization quality, combined with a temporal penalty. This design reduces the reliance on dense box annotations and enables training directly on standard video QA data. On the V-STAR benchmark, SER achieves 49.6
Zero-shot Dialog State Tracking (zs-DST) is essential for enabling Task-Oriented Dialog Systems (TODs) to generalize to new domains without costly data annotation. A central challenge lies in the semantic misalignment between dynamic dialog contexts and static prompts, leading to inflexible cross-layer coordination, domain interference, and catastrophic forgetting. To tackle this, we propose Hierarchical Collaborative Low-Rank Adaptation (HiCoLoRA), a framework that enhances zero-shot slot inference through robust prompt alignment. It features a hierarchical LoRA architecture for dynamic layer-specific processing (combining lower-layer heuristic grouping and higher-layer full interaction), integrates Spectral Joint Domain-Slot Clustering to identify transferable associations (feeding an Adaptive Linear Fusion Mechanism), and employs Semantic-Enhanced SVD Initialization (SemSVD-Init) to preserve pre-trained knowledge. Experiments on multi-domain datasets MultiWOZ and SGD show that HiCoLoRA outperforms baselines, achieving SOTA in zs-DST. Code is available at https://github.com/carsonz/HiCoLoRA.
Accurate coronary Digital Subtraction Angiography (DSA) vessel segmentation is essential for computer-aided diagnosis and treatment planning of coronary artery disease (CAD). However, thin low-contrast vessels, background interference, and severe vessel-background class imbalance make reliable segmentation of weak distal branches and vessel boundaries challenging. Existing methods struggle to balance global contextual reasoning with preservation of weak vessels, vessel continuity, and fine boundaries. To address these limitations, we propose HTC-SGA Former, a lightweight hybrid Transformer-CNN framework for coronary DSA vessel segmentation. It employs a CNN encoder for local vessel morphology extraction and a Transformer decoder for contextual feature modeling. A Multi-Scale Global-Local Window Attention (MS-GLWA) block performs efficient global-local contextual modeling, while a Self-Guided Feature Attention (SGFA) module enhances weak-vessel responses. In addition, a Boundary-Weighted Adaptive Compound Loss (BWACL) emphasizes thin-vessel boundaries and adaptively balances vessel recovery and boundary refinement. Experiments on private right and left coronary artery DSA subsets show that HTC-SGA Former outperforms 14 state-of-the-art segmentation methods while maintaining a compact architecture with only 0.81M parameters. BWACL also improves performance over binary cross-entropy and Dice losses across four encoder-decoder architectures, demonstrating strong cross-backbone applicability. HTC-SGA Former improves thin-vessel recovery, vessel continuity, and boundary localization through complementary global-local contextual modeling, vessel-focused refinement, and adaptive optimization, supporting reliable and computationally efficient coronary vessel analysis for future computer-assisted cardiovascular interventions.
Anatomical structure masks are widely adopted in radiotherapy dose prediction, as they provide explicit geometric constraints that facilitate structure-dose coupling. However, conventional manual delineation of these masks requires precise annotation of structure boundaries relevant to radiotherapy, which is time-consuming and labor-intensive. To address these limitations, we propose a scribble-guided dose prediction framework that relies solely on anatomical structures annotated with sparse scribbles. Specifically, we design a Scribble Completion Module (SCM) to generate dense anatomical masks by propagating sparse scribble labels to semantically similar voxels. During the propagation process, a supervoxel-based regularization is introduced to preserve geometric boundary consistency to ensure anatomical plausibility. Furthermore, we propose a Structure-Guided Dose Generation Module (SGDGM) to strengthen the correspondence between sparse structural cues and dose distribution. The completed dense masks derived from scribbles serve as structural guidance to condition dose prediction, forming a scribble-mask-dose learning pipeline under sparse annotation. Experiments on the GDP-HMM dataset demonstrate that ScribbleDose achieves competitive dose prediction performance using only sparse structural annotations. The source code and reannotated scribble annotations are publicly available at https://github.com/iCherishxixixi/ScribbleDose.
Accurate segmentation of aortic vascular structures is critical for diagnosing and treating cardiovascular diseases. Traditional Transformer-based models have shown promise in this domain by capturing long-range dependencies between vascular features. However, their reliance on fixed-size rectangular patches often influences the integrity of complex vascular structures, leading to suboptimal segmentation accuracy. To address this challenge, we propose the adaptive Morph-Patch Transformer (MPT), a novel architecture specifically designed for aortic vascular segmentation. Specifically, MPT introduces an adaptive patch partitioning strategy that dynamically generates morphology-aware patches aligned with complex vascular structures. This strategy can preserve semantic integrity of complex vascular structures within individual patches. Moreover, a Semantic Clustering Attention (SCA) method is proposed to dynamically aggregate features from various patches with similar semantic characteristics. This method enhances the model's capability to segment vessels of varying sizes, preserving the integrity of vascular structures. Extensive experiments on three open-source datasets (AVT, AortaSeg24 and TBAD) demonstrate that MPT achieves state-of-the-art performance, with improvements in segmenting intricate vascular structures.
Traditional task-oriented dialog systems are unable to evolve from ongoing interactions or adapt to new domains after deployment, that is a critical limitation in real-world dynamic environments. Continual learning approaches depend on episodic retraining with human curated data, failing to achieve autonomy lifelong improvement. While evolutionary computation and LLM driven self improvement offer promising mechanisms for dialog optimization, they lack a unified framework for holistic, iterative strategy refinement. To bridge this gap, we propose DarwinTOD, a lifelong self evolving dialog framework that systematically integrates these two paradigms, enabling continuous strategy optimization from a zero-shot base without task specific fine-tuning. DarwinTOD maintains an Evolvable Strategy Bank and operates through a dual-loop process: online multi-agent dialog execution with peer critique, and offline structured evolutionary operations that refine the strategy bank using accumulated feedback. This closed-loop design enables autonomous continuous improvement without human intervention. Extensive experiments show that DarwinTOD surpasses previous state-of-the-art methods and exhibits continuous performance gains throughout evolution. Our work provides a novel framework for building dialog systems with lifelong self evolution capabilities.
Automatic medical image segmentation is of significant value in clinical assisted diagnosis. However, complex model architectures severely restrict their deployment in primary medical institutions with limited computational resources. The excessive compression of model capacity during the design of existing lightweight networks often leads to problems such as degraded local orientation perception, sparsified multi-scale receptive fields, and cross-layer semantic contamination, rather than independent performance declines. To address these limitations, this work proposes RWS-Net, a lightweight segmentation framework that improves representational efficiency per unit of computational cost by synergistically reconfiguring the structural inductive bias, spatial support scope, and cross-layer information flow of convolutions. Concretely, the network decouples and recombines multi-order directional gradient operators with spatial features, explicitly embedding spatial gradient priors to enhance local gradient variations and directional structural sensitivity, while structural re-parameterization during inference reduces computational overhead. Furthermore, complementary integration of asymmetric and dilated convolutions expands the effective receptive field within a constrained parameter budget and alleviates local response discontinuity induced by sparse sampling. A coordinated gated fusion module is devised to bidirectionally filter and integrate encoded details and decoded semantics, thereby resolving semantic discrepancies across multi-scale representations. RWS-Net achieves highly competitive segmentation accuracy across eleven tasks with only 0.509 million parameters and 1.388G flops of computational cost. Our code is available at https://github.com/qinxin99/RWS-Net.
Vision-and-Language Navigation (VLN) requires an embodied agent to navigate in a complex 3D environment according to natural language instructions. Recent progress in large language models (LLMs) has enabled language-driven navigation with improved interpretability. However, most LLM-based agents still rely on single-shot action decisions, where the model must choose one option from noisy, textualized multi-perspective observations. Due to local mismatches and imperfect intermediate reasoning, such decisions can easily deviate from the correct path, leading to error accumulation and reduced reliability in unseen environments. In this paper, we propose DV-VLN, a new VLN framework that follows a generate-then-verify paradigm. DV-VLN first performs parameter-efficient in-domain adaptation of an open-source LLaMA-2 backbone to produce a structured navigational chain-of-thought, and then verifies candidate actions with two complementary channels: True-False Verification (TFV) and Masked-Entity Verification (MEV). DV-VLN selects actions by aggregating verification successes across multiple samples, yielding interpretable scores for reranking. Experiments on R2R, RxR (English subset), and REVERIE show that DV-VLN consistently improves over direct prediction and sampling-only baselines, achieving competitive performance among language-only VLN agents and promising results compared with several cross-modal systems.Code is available at https://github.com/PlumJun/DV-VLN.
Recent large language model (LLM)-based video summarization methods score frame importance from generated captions, but they often suffer from a language bottleneck, where fine-grained visual evidence and temporal cues may be weakened during caption generation. To address this issue, we propose a visual calibration framework for text-guided video summarization. The proposed method keeps the text branch as the primary semantic scoring pathway, while introducing temporally modeled visual features as complementary calibration signals. Specifically, visual evidence is incorporated through cross-attention, global visual context modeling, and a gated fusion mechanism for controlled score refinement. Experiments on SumMe and TVSum demonstrate that the proposed method effectively compensates for the visual information lost in captionbased scoring and achieves more accurate and stable frame-level importance prediction.
Recent advances in vision-language cross-modal learning have substantially improved the performance of video temporal grounding. However, most existing methods directly associate global video features with sentence-level features, overlooking the fact that textual semantics usually correspond to only limited spatio-temporal regions within a video. This limitation often leads to unstable alignment in complex scenarios involving intertwined events and diverse actions. In essence, accurate video temporal grounding requires the joint modeling of fine-grained spatial semantics and heterogeneous temporal event structures. Motivated by this observation, we propose a hierarchical prototype alignment approach that models cross-modal correspondence between video and text through structured intermediate prototype representations. Specifically, the alignment process is decomposed into two complementary stages: object-phrase alignment and event-sentence alignment. In the object-phrase alignment stage, discriminative local visual regions and informative textual words are aggregated to construct object and phrase prototypes, thereby enhancing fine-grained spatial correspondence at the level of entities and localized actions. In the event-sentence alignment stage, object prototypes are further integrated along the temporal dimension to form event prototypes that represent continuous action units, enabling effective alignment with sentence-level semantics and facilitating the modeling of diverse temporal event structures. On this basis, we further directly inject cross-modal alignment information into candidate moment aggregation. This design allows candidate moment representations to emphasize query-relevant temporal regions. Extensive experiments on Charades-STA, ActivityNet Captions, and TACoS demonstrate that the proposed method outperforms existing approaches, validating the effectiveness of hierarchical prototype alignment for improving both cross-modal alignment quality and temporal grounding accuracy.
The task of temporal answer grounding in instructional videos (TAGV), which aims to locate precise video segments that respond to natural language queries, is increasingly important for direct video answer retrieval. This task remains challenging due to the need to comprehend semantically complex questions and to address the significant length mismatch between untrimmed videos and short target moments. Existing methods often suffer from sensitivity to irrelevant content or insufficient visual reasoning capabilities. To tackle these limitations, we propose a Candidate-Aware Causal Reasoning (CACR) framework. Our approach first employs a Visual-Language Pre-training based Candidate Selection (VBCS) algorithm to efficiently generate K candidate segments, then applies a temporal logic reasoning module enhanced by a rejection reward mechanism and optimized via Group Relative Policy Optimization (GRPO) for robust inference. Extensive experiments on six benchmarks demonstrate that our method achieves state-of-the-art performance in terms of mean Intersection-over-Union (mIoU), providing a new perspective for reasoning-based retrieval in long videos.
Large Vision-Language Models (LVLMs) often suffer from object hallucination, generating text inconsistent with visual inputs, which can critically undermine their reliability. Existing inference-time interventions to mitigate this issue present a challenging trade-off: while methods that steer internal states or adjust output logits can be effective, they often incur substantial computational overhead, typically requiring extra forward passes. This efficiency bottleneck can limit their practicality for real-world, latency-sensitive deployments. In this work, we aim to address this trade-off with Residual-Update Directed DEcoding Regulation (RUDDER), a low-overhead framework that steers LVLMs towards visually-grounded generation. RUDDER is built on two key innovations: (1) Contextual Activation Residual Direction (CARD) vector, a per-sample visual evidence vector extracted from the residual update of a self-attention layer during a single, standard forward pass. (2) A Bayesian-inspired adaptive gate that performs token-wise injection, applying a corrective signal whose strength is conditioned on the model's deviation from the visual context. Extensive experiments on key hallucination benchmarks, including POPE and CHAIR, indicate that RUDDER achieves performance comparable to state-of-the-art methods while introducing negligible computational latency, validating RUDDER as a pragmatic and effective approach for improving LVLMs' reliability without a significant compromise on efficiency.
Following the CMIVQA, MMI-VQA, and M4IVQA challenges in NLPCC 2023–2025, we introduce the Difficulty-Aware Medical Instructional Video Question Answering (DA-MIVQA) shared task for NLPCC 2026. DA-MIVQA extends previous multilingual and multimodal medical video benchmarks by explicitly distinguishing questions according to the type and complexity of evidence required for answering. Specifically, simple questions can often be answered from subtitle-based textual cues, whereas complex questions require visual grounding, procedural understanding, and cross-modal evidence integration. The challenge contains three tracks: Difficulty-Aware Temporal Answer Grounding in Single Video (DA-TAGSV), Difficulty-Aware Video Corpus Retrieval (DA-VCR), and Difficulty-Aware Temporal Answer Grounding in Video Corpus (DA-TAGVC). The dataset is collected from public medical instructional channels, covers diverse scenarios such as first aid, emergency response, rehabilitation, nursing, and general medical education, and is manually verified with difficulty annotations. This paper presents the task motivation, dataset construction, evaluation protocol, participation overview, competition results, and representative systems of DA-MIVQA. DA-MIVQA provides a practical benchmark for evaluating medical instructional video question answering systems under varying textual, visual, temporal, and procedural reasoning requirements.
Vision-Language Action (VLA) models have shown remarkable progress in robotic manipulation by leveraging the powerful perception abilities of Vision-Language Models (VLMs) to understand environments and directly output actions. However, by default, VLA models may overly attend to image tokens in the task-irrelevant region, which we describe as 'distracting tokens'. This behavior can disturb the model from the generation of the desired action tokens in each step, affecting the success rate of tasks. In this paper, we introduce a simple yet effective plug-and-play Distracting Token Pruning (DTP) framework, which dynamically detects and prunes these distracting image tokens. By correcting the model's visual attention patterns, we aim to improve the task success rate, as well as exploring the performance upper boundaries of the model without altering its original architecture or adding additional inputs. Experiments on the SIMPLER Benchmark (Li et al., 2024) show that our method consistently achieving relative improvements in task success rates across different types of novel VLA models, demonstrating generalizability to transformer-based VLAs. Further analysis reveals a negative correlation between the task success rate and the amount of attentions in the task-irrelevant region for all models tested, highlighting a common phenomenon of VLA models that could guide future research. We also publish our code at: https://anonymous.4open.science/r/CBD3.
Accurate segmentation of coronary Digital Subtraction Angiography (DSA) images is essential to diagnose and treat coronary artery diseases (CAD). Despite advances in deep learning, challenges such as high intra-class variance and class imbalance limit precise vessel delineation. Most existing approaches for coronary DSA segmentation cannot address these issues. Furthermore, existing segmentation networks’ encoders do not directly generate semantic embeddings, which could enable the decoder to reconstruct segmentation masks effectively from these well-defined features. We propose a Supervised Prototypical Contrastive Loss (SPCL) that combines supervised and prototypical contrastive learning to enhance coronary DSA image segmentation. The supervised contrastive loss enforces semantic embeddings in the encoder, improving feature differentiation. The prototypical contrastive loss allows the model to focus on the foreground class while alleviating the high intra-class variance and class imbalance problems by concentrating only on the hard-to-classify background samples. We implement the proposed SPCL loss within an MSA-UNet3+: a Multi-Scale Attention-Enhanced UNet3+ architecture. The architecture integrates key components: a Multi-Scale Attention Encoder (M-encoder) and a Multi-Scale Dilated Bottleneck (MSD-Bottleneck) designed to enhance multi-scale feature extraction and a Contextual Attention Fusion Module (CAFM) designed to preserve fine-grained details while improving contextual understanding. Experiments on a private coronary DSA dataset demonstrate that MSA-UNet3+ outperforms state-of-the-art methods, achieving the highest Dice coefficient and F1-score and significantly reducing ASD and ACD. The developed framework provides clinicians with precise vessel segmentation, enabling accurate identification of coronary stenosis and supporting informed diagnostic and therapeutic decisions. The code will be released at https://github.com/rayanmerghani/MSA-UNet3plus.
Guide-wire segmentation is of great significance for robotic vascular interventions. However, due to its curvilinear structure and weak contrast with the image background, the accurate detection of the guide-wire has become a major challenge. In recent years, many general semantic segmentation models have been widely used for guide-wire segmentation, and some studies have used endpoint detection or multi-part position detection of the guide-wire to reduce the difficulty of segmentation, but ignored the morphological features of the guide-wire. In our work, we propose an end-to-end guide-wire segmentation method that incorporates multiple morphological features into neural network to enhance the morphological features of the guide-wire and improve segmentation accuracy. Specifically, we adopt a Ushaped encoder-decoder framework as the backbone network for the overall feature extraction. We then introduce the Tubular Feature Extraction Module (TFEM), designed for elongated tubular objects to enhance the learning of guide wire tubular features. Next, we introduce the Feature Extension Module (FEM) to improve the combining ability of the extracted tubular features. Finally, we devise a Composite Edge Loss (CEL) that focuses more on edge features to further optimize the model's effectiveness in edge details. Our method achieves better guidewire segmentation results for morphological details, achieving state-of-the-art performance on 3669 acquired multi-class X-ray guide-wire images.
In the field of medical image processing, registration algorithms are crucial tools, especially in assisting physicians with aligning medical images acquired at different time points or through different modalities. These techniques are particularly important for medical applications such as disease diagnosis, lesion detection, surgical planning, and treatment monitoring. However, although most deep learning-based methods are capable of extracting multiscale features, they may fail to produce outputs that are directly related to the fnal deformation feld. Additionally, many methods based on the U-Net structure overly rely on the last layer of high-resolution images, which represents a significant drawback. To address these issues, we propose a novel unsupervised deformable registration method named FocusMorph. This method centers on the FLatten Transformer block and employs a focused linear attention mechanism to enhance attentional expressivity while maintaining low complexity. We have also designed a layer-by-layer output fusion mechanism and a motion image encoder specifically for medical image registration, which aids in continuously tracking positional differences between motion images and effectively fusing them. Experimental results indicate that the FocusMorph method surpasses current leading medical image registration techniques on two distinct brain image datasets. It achieves improvements in the Dice coefficient by 2.6% and 1.5%, respectively, confirming its superior performance and significant potential in image registration. These findings not only highlight FocusMorph's robust registration capabilities but also underscore its promising prospects in medical image processing.
While large language models (LLMs) have demonstrated remarkable potential in time series forecasting, their practical deployment remains constrained by excessive computational demands and memory footprints. Existing LLM-based methods typically suffer from three critical limitations: (1) inefficient parameter utilization in handling numerical time series patterns; (2) modality misalignment between continuous temporal signals and discrete text embeddings; and (3) inflexibility for real-time expert knowledge integration. We present small but mighty enhancing time series (SMETimes), the first systematic investigation of small language models with sub-3B parameters (SLM) for efficient and accurate time series forecasting. Our method centers on three key innovations: (1) a statistically enhanced prompt structure that bridges numerical time series with textual semantics through descriptive statistical features; (2) an adaptive fusion embedding structure that aligns temporal patterns with language model token spaces through learnable parameters; and (3) a dynamic mixture-of-experts structure enabled by SLMs’ computational efficiency, adaptively combining base predictions with domain-specific models. Extensive evaluations across seven benchmark datasets (ETTh1/2, ETTm1/2, Weather, Solar, ECL) demonstrate that our 3B-parameter SLM achieves state-of-the-art performance on five primary datasets while maintaining 3.8 × faster training and 5.2 × lower memory consumption compared to 7B-parameter LLM baselines. In particular, the proposed model exhibits better learning capabilities, achieving 12.3 https://github.com/xiyan1234567/SMETimes .
Medical Visual Question Answering (Med-VQA) represents a critical and challenging subtask within the general VQA domain. Despite significant advancements in general VQA, multimodal large language models (MLLMs) still exhibit substantial limitations when handling multi-task VQA scenarios. These limitations manifest through erroneous spatial localization and misinterpretation of medical images, which primarily arise from two fundamental issues: inadequate image-text alignment and insufficient domain-specified knowledge for medical applications. To address these issues, we introduce the Cross-Modal Clinical Knowledge Distiller (ClinKD), an innovative framework designed to enhance image-text alignment and establish more effective medical knowledge transformation mechanisms, which enables MLLMs to perform better even when lacking prior medical knowledge. Our extensive experimental evaluations demonstrate that the ClinKD achieves state-of-the-art performance on several datasets which are challenging for Med-VQA task. The results indicate that our approach not only significantly improves image-text alignment but also effectively enables MLLMs to adapt to the medical knowledge. The source code for ClinKD is available at: https://github.com/overloadedHenry/ClinKD.
Three-dimensional blood vessel segmentation is an important and challenging task that faces two main difficulties: (1) blood vessel structures are small, making them hard to capture by the network, and vessel edges are difficult to segment accurately; (2) false positives are prone to occur due to the presence of artifacts and noise. This paper proposes a novel blood vessel segmentation method called MambaVesselNet. This method is based on a state-space model and employs a selective state-space time series modeling strategy to achieve a larger receptive field. To better capture fine vascular structures and accurately segment edges, this paper introduces an edge enhancement module and a feature selection module. In terms of data preprocessing, nnUNet's preprocessing strategy is adopted to ensure spatial consistency of the input data. Evaluation on three standard vascular segmentation benchmarks shows that MambaVesselNet achieves state-of-the-art performance. Specifically, on cardiovascular and liver vessel datasets, the Dice coefficient is improved by 1.38% and 2.69%, respectively. The contributions of this paper include the proposal of a new module for enhancing blood vessel edge features, the development of a feature selection module with long sequence modeling capability, and the adoption of nnUNet's data preprocessing strategy, setting a new benchmark for blood vessel segmentation technology.