Audio-Visual Instance Segmentation (AVIS) aims to simultaneously classify, segment, and track sounding objects within video sequences. Unlike Audio-Visual Semantic Segmentation (AVS), AVIS involves instance-level modeling across longer video sequences, introducing two key challenges: (1) complex modality-state changes disrupt long-range modeling, and (2) substantial structural and distributional discrepancies between modalities hinder precise instance-level association. Existing methods rely on fixed-step Transformers and recursive Mamba models, lacking adaptability to modality-state changes. In addition, methods performing implicit matching ignore the inherent distributional inconsistencies. To address these issues, we propose a framework with Adaptive Dynamic Step Modulation (ADSM) and Optimal Transport-based Matching Modulation (OT-MM). ADSM adaptively modulates Mamba step sizes using temporal variation, cross-modal discrepancy, and historical context, balancing rapid response to modality-state changes with stable long-range modeling. OT-MM explicitly formulates instance-level cross-modal matching as an entropy-regularized optimal transport problem solved via log-domain Sinkhorn iterations, and further enforces distribution-level coherence with an MMD regularizer. Extensive experiments demonstrate state-of-the-art performance on the AVIS benchmark (+3.76 FSLA, +2.75 HOTA, +2.58 mAP), verified through comprehensive qualitative visualizations. The code and model are available at https://github.com/happylife-pk/SAMOT.

Image exposure correction tasks often involve addressing issues such as overexposure, underexposure, and uneven exposure. These exposure errors can result in loss of image details, color distortion, and significant degradation of image quality. Exposure correction can be divided into two main challenges: recovering structural information like details and colors, and restoring proper illumination levels. To tackle these challenges more effectively, we propose a dual-prompt approach that introduces structural prompts and illumination prompts to guide the network’s learning process. (i) Structural prompts consist of a set of learnable parameters that help the network adopt specific restoration strategies based on different lighting conditions. For varying degrees of overexposed and underexposed inputs, structural prompts guide the network to apply the appropriate structural restoration strategies to better preserve and enhance image details. (ii) Illumination prompts are divided into positive text prompts and negative text prompts. Negative prompts are further categorized into overexposure negative prompts and underexposure negative prompts. We use CLIP (Contrastive Language–Image Pretraining) to align text prompts with images, creating loss functions that accurately assess the exposure state of the image. Compared to directly inputting text descriptions into CLIP, our pre-trained, learnable text prompt parameters more robustly guide the loss function to make precise assessments of the input image’s exposure quality. In summary, our dual-prompt approach effectively addresses the key issues of detail recovery and illumination estimation in exposure correction tasks through the combined action of structural and illumination prompts.

Multi-scale feature extraction has played a crucial role in advancing RGB-based salient object detection. Many studies have achieved significant progress by designing sophisticated modules to strengthen multi-scale representations. Nevertheless, existing approaches typically employ fixed receptive fields, which is an inappropriate way to capture objects at different scales. Additionally, achieving balanced integration of low-level and high-level features during fusion remains a persistent challenge in developing efficient SOD architectures. To address these problems, we proposed a hierarchical scalable receptive fields search framework consisting of a novel searchable cell and a scene-aware search space. The proposed cell is able to ease the inconsistency between the sizes of receptive field and multi-scale features with an affordable budget. In addition, the search space explores the relationship between the contribution and the number of channels of multi-level features, exploring the appropriate number of the proposed cell for an efficient SOD model. Comprehensive evaluations on five benchmark datasets confirm the proposed method’s effectiveness, showing competitive performance against state-of-the-art approaches under four evaluation metrics. The source code will be publicly available at https://github.com/LiuTingWed/HSRF-SOD .

Accurately modeling complex temporal and topological dependencies and depth information is critical for monocular 3D human pose estimation, yet existing Mamba-based approaches struggle to fulfill these demands, suffering from internal state update confusion induced by forced sequence flattening and depth modeling confusion inherent to single-view observations. To address this confusion, we propose a StructMamPose framework equipped with Identity Anchoring Mechanism (IAM) and View Transformation Hub (VTH). The IAM injects spatiotemporal identities into the parameter generation network to anchor the selectivity of state update matrices, suppressing spurious connections to enforce feature propagation along valid topological dependencies. The VTH performs internal coordinate rotation to transform implicit depth inference into observable planar features, endowing the model with explicit spatial understanding and multi-view constraints. Experimental results demonstrate that our framework achieves SOTA performance on most datasets.

Perturbation-based semi-supervised learning has shown promising performance in medical image segmentation by exploiting abundant unlabeled data. However, existing interpolation-based perturbation strategies often rely on fixed or random perturbation ratios, which may introduce excessive perturbations, unstable learning, and boundary inaccuracies caused by contextual discontinuities at cropped regions. To address these issues, we propose P3Net, a progressive and periodic perturbation framework for semi-supervised medical image segmentation. Specifically, we propose a Progressive and Periodic Perturbation Mechanism (P3M) that dynamically adjusts the interpolation ratio during training, enabling the model to gradually adapt to perturbations from labeled-to-unlabeled data in a cyclic manner. In addition, we design a boundary-focused loss to explicitly emphasize error-prone boundary regions introduced by hard-cropping interpolation, thereby improving the model's sensitivity to local structural details. We evaluate the proposed method on two 2D datasets and two 3D datasets under low-label settings. Experimental results show that P3Net achieves competitive or superior performance compared with recent semi-supervised segmentation methods. For example, on the Pancreas-NIH dataset with only 6% labeled data, P3Net achieves a Dice score of 80.01%, corresponding to a relative improvement of 4.59% over the previous best result. Further ablation studies demonstrate the effectiveness of both P3M and the boundary-focused loss, and additional experiments indicate that the proposed perturbation mechanism can be incorporated into existing interpolation-based methods to improve their performance. These results suggest that progressively regulated perturbation is an effective strategy for improving semi-supervised medical image segmentation under limited annotation scenarios.

Multimodal survival analysis utilizing whole slide images (WSIs) and genomic profiles is fundamental for cancer prognosis. Recently, state-space models like Mamba have emerged as powerful tools for sequence modeling. However, translating this success to complex multimodal tasks is hindered by two critical limitations. First, conventional fusion strategies assume a static multimodal interaction strength, ignoring the fluctuating diagnostic importance of each modality across different patients and local regions. Second, the standard Mamba architecture processes tokens along predefined physical paths. This rigid scanning disrupts the semantic continuity of spatially scattered medical features and exacerbates long-range decay. To address these challenges, we introduce AdaSurvMamba as a novel adaptive framework for multimodal survival analysis. The framework features a Dual-Scale Importance-Aware Reconstruction (DSIR) module to dynamically modulate cross-modal interaction strength. It evaluates diagnostic importance at both the sequence and token levels to reconstruct the input representations. Furthermore, we propose a Semantic Aggregation Scanning (SAS) module to overcome contextual fragmentation. The SAS module dynamically reorganizes discrete tokens into semantically continuous sequences via a shared prototype pool. It explicitly modulates the state transition step size using global modality context and semantic priors to adaptively control the information absorption rate. Experiments across five TCGA cohorts demonstrate consistent gains over existing methods. Code is available at https://github.com/zjlGO/AdaSurvMamba.

Electrocardiography (ECG) is a widely adopted modality for monitoring cardiac rate and rhythm and identifying various abnormalities of the cardiac electrical system. Research on ECG preprocessing and intelligent classification is vital for the early diagnosis and clinical management of cardiovascular diseases and helps improve the accuracy and efficiency of cardiac diagnosis. Nevertheless, ECG signals captured by wearable acquisition devices frequently suffer from poor signal quality due to subject motion and uncontrolled acquisition environments. Furthermore, native ECG waveforms are faint and susceptible to diverse noise interference. To diagnose noise-corrupted arrhythmia, clinicians need to inspect lengthy ECG recordings relying on professional expertise, which is tedious, labor-intensive and prone to subjective bias. Another critical challenge lies in severe class imbalance within public ECG datasets, resulting in degraded classification performance for minority arrhythmia categories. Accordingly, developing robust automated arrhythmia detection algorithms is essential to ease the diagnostic workload of clinical practitioners. In this work, discrete wavelet transform (DWT) is adopted for noise elimination, and the SMOTE-Tomek hybrid resampling strategy is applied to mitigate class imbalance on the MIT-BIH and INCART datasets. We further propose MSCA-TNet, a hybrid architecture embedding the adaptive channel attention (ACA) module and Transformer to extract both local fine-grained features and global long-range contextual information from ECG sequences. The ACA module dynamically modulates channel-wise feature weights to emphasize clinically discriminative features while suppressing irrelevant components. On the two benchmark datasets, the proposed model yields macro-average accuracies of 93.97% and 98.79%, alongside overall accuracies of 98.88% and 99.47%.

Segment Anything Model 3 (SAM3) advances open-vocabulary segmentation through promptable concept segmentation, enabling users to segment all instances associated with a given concept using short noun-phrase (NP) prompts. While effective for concept-level grounding, real-world interactions often involve far richer natural-language instructions that combine attributes, relations, actions, states, or implicit reasoning. Currently, SAM3 relies on external multi-modal agents to convert complex instructions into NPs and conducts iterative mask filtering, leading to coarse representations and limited instance specificity. In this work, we present SAM3-I, an instruction-following extension of the SAM family that unifies concept-level grounding and instruction-level reasoning within a single segmentation framework. Built upon SAM3, SAM3-I introduces an instruction-aware cascaded adaptation mechanism with dedicated alignment losses that progressively aligns expressive instruction semantics with SAM3's vision-language representations, enabling direct interpretation of natural-language instructions while preserving its strong concept recall ability. To enable instruction-following learning, we introduce HMPL-Instruct, a large-scale instruction-centric dataset that systematically covers hierarchical instruction semantics and diverse target granularities. Experiments demonstrate that SAM3-I achieves appealing performance across referring and reasoning-based segmentation, showing that SAM3 can be effectively extended to follow complex natural-language instructions without sacrificing its original concept-driven strengths. Code and dataset are available at https://github.com/debby-0527/SAM3-I.

Accurate organ segmentation is essential for medical image analysis. Semi-supervised medical image segmentation reduces annotation costs while maintaining high segmentation precision. However, it still suffers from distribution shift between labeled and unlabeled data and severe class imbalance, which degrades pseudo-label quality and severely hinders segmentation of critical targets. To address these issues, we propose a novel semi-supervised framework integrating two key strategies. Firstly, we propose a Class-Sensitive Temperature Scaling (CSTS) strategy that dynamically calibrates logit adjustment by modeling class-wise consistency and confusion, applying both global and local regulation, and leveraging a dual-head decoupled architecture for robust per-class adaptation to distribution shift. Secondly, we introduce a Discrepancy-Aware Sampling Strategy (DSS) that forms a closed-loop feedback system to guide conditional diffusion models in generating high-quality samples with enhanced representations for minority classes, boosting segmentation performance. Experiments on two public abdominal multi-organ segmentation datasets demonstrate that our method outperforms state-of-the-art techniques, achieving comprehensive improvements in segmentation accuracy with particularly significant gains for minority classes. The code of our method is available at https://github.com/LiuTingWed/C2DS LiuTingWed/C2DS.

In computational pathology, Hematoxylin and Eosin (H&E) staining offers a cost-effective solution for tissue analysis, while Immunohistochemistry (IHC) delivers specific biomarker expression at substantially higher cost and operational complexity. Existing H&E-to-IHC translation methods predominantly operate at the pixel level, often overlooking the preservation of high-level semantic features required by modern multi-instance learning frameworks. To bridge this gap, we present FeatStainDiff, a diffusion-based model that performs direct feature-level transformation between staining modalities. Our framework incorporates two novel components: a Contrastive Semantic Bridging mechanism that ensures diagnostic semantics are preserved during cross-modal translation, and a Frequency-domain Mixture of Experts module that adaptively handles distribution shifts through spectral processing. This design enables the generation of high-fidelity and pathologically consistent IHC features directly from H&E inputs. Through extensive evaluation on two virtual staining datasets and two whole-slide image classification benchmarks, we demonstrate that FeatStainDiff consistently surpasses existing approaches. The method achieves significant improvements in feature similarity metrics, while downstream classification tasks benefit from markedly enhanced performance. FeatStainDiff provides an effective and practical pathway for computational biomarker prediction, with promising potential to expand access to specialized staining analysis in resource-limited clinical environments. Code will be made publicly available upon publication.

The ability to capture and segment sounding objects in dynamic visual scenes is crucial for the development of Audio-Visual Segmentation (AVS) tasks. While significant progress has been made in this area, the interaction between audio and visual modalities still requires further exploration. In this work, we aim to answer the following questions: How can a model effectively suppress audio noise while enhancing relevant audio information? How can we achieve discriminative interaction between the audio and visual modalities? To this end, we propose SDAVS, equipped with the Selective Noise-Resilient Processor (SNRP) module and the Discriminative Audio-Visual Mutual Fusion (DAMF) strategy. The proposed SNRP mitigates audio noise interference by selectively emphasizing relevant auditory cues, while DAMF ensures more consistent audio-visual representations. Experimental results demonstrate that our proposed method achieves state-of-the-art performance on benchmark AVS datasets, especially in multi-source and complex scenes. The code and model are available at https://github.com/happylife-pk/SDAVS.

Predicting river flow in places without streamflow records is challenging because basins respond differently to climate, terrain, vegetation, and soils. Traditional basin attributes describe some of these differences, but they cannot fully represent the complexity of natural environments. This study examines whether AlphaEarth Foundation embeddings, which are learned from large collections of satellite images rather than designed by experts, offer a more informative way to describe basin characteristics. These embeddings summarize patterns in vegetation, land surface properties, and long-term environmental dynamics. We find that models using them achieve higher accuracy when predicting flows in basins not used for training, suggesting that they capture key physical differences more effectively than traditional attributes. We further investigate how selecting appropriate donor basins influences prediction in ungauged regions. Similarity based on the embeddings helps identify basins with comparable environmental and hydrological behavior, improving performance, whereas adding many dissimilar basins can reduce accuracy. The results show that satellite-informed environmental representations can strengthen hydrological forecasting and support the development of models that adapt more easily to different landscapes.

Edge-assisted mobile video analytics (MVA) applications are increasingly shifting from using vision models based on convolutional neural networks (CNNs) to those built on vision transformers (ViTs) to leverage their superior global context modeling and generalization capabilities. However, deploying these advanced models in latency-critical MVA scenarios presents significant challenges. Unlike traditional CNN-based offloading paradigms where network transmission is the primary bottleneck, ViT-based systems are constrained by substantial inference delays, particularly for dense prediction tasks where the need for high-resolution inputs exacerbates the inherent quadratic computational complexity of ViTs. To address these challenges, we propose a dynamic mixed-resolution inference strategy tailored for ViT-backboned dense prediction models, enabling flexible runtime trade-offs between speed and accuracy. Building on this, we introduce ViTMAlis, a ViT-native device-to-edge offloading framework that dynamically adapts to network conditions and video content to jointly reduce transmission and inference delays. We implement a fully functional prototype of ViTMAlis on commodity mobile and edge devices. Extensive experiments demonstrate that, compared to state-of-the-art accuracy-centric, content-aware, and latency-adaptive baselines, ViTMAlis significantly reduces end-to-end offloading latency while improving user-perceived rendering accuracy, providing a practical foundation for next-generation mobile intelligence.

Audio-visual instance segmentation (AVIS) requires accurately identifying and tracking individual sounding objects with pixel-level masks. Existing methods struggle to match overlapping acoustic events with visual instances and handle asynchronous audio-visual dynamics. Therefore, two critical questions arise: how can a model establish precise correspondence between overlapping sound sources and visual instances, and how can a model maintain robust tracking when audio and visual signals are temporally misaligned?This paper proposes Hear to See (H2S), addressing these challenges through two mechanisms. The Acoustic-Semantic Projector (ASP) disentangles mixed audio and establishes hierarchical correspondence from semantic to spatial domains. The Asynchronous Dynamics Modulator (ADM) adaptively adjusts state transitions via audio-modulated Mamba, prioritizing current information during dynamic variations and maintaining continuity in stable periods.Experiments on AVISeg show H2S achieves SOTA performance, attaining 48.54 mAP with a COCO pretrained ResNet50 and surpassing the previous by 7.8%. The code will be open-sourced once the paper is accepted. The source code will be publicly available at https://github.com/leiyeliu/H2S.

Echocardiography, a vital cardiac imaging modality, faces challenges due to limited annotated data, impeding the application of deep learning. This paper introduces EchoCardMAE, a customized masked video autoencoder framework designed to leverage unlabeled echocardiography data and enhance performance across diverse cardiac tasks. EchoCardMAE addresses key challenges in echocardiogram analysis through three innovations built upon masked video modeling (MVM): (1) Key Area Masking, which concentrates feature learning on the diagnostically relevant sector of the image; (2) Temporal-Invariant Alignment Loss, promoting feature consistency across different clips of the same echocardiogram; and (3) Reconstruction Denoising, improving robustness to speckle noise inherent in echocardiography. We comprehensively evaluated EchoCardMAE on three public datasets, demonstrating stateof-the-art results in ejection fraction (EF) estimation, Myocardial infarction (MI) prediction, and cardiac segmentation. For example, on the EchoNet-Dynamic dataset, EchoCardMAE achieved an EF estimation MAE of 3.78 and a left ventricular segmentation mDice of 92.96, surpassing existing methods. The code is available at https://github.com/ m1dsolo/EchoCardMAE.

The precise and reliable image detection methods (object detection) play a pivotal role in the early diagnosis of gastrointestinal cancer lesions. However, existing related approaches are limited in extracting multiscale features and inadequately labeled datasets. This study presents a semi-supervised polyp detection via a target-aware image reinforcement strategy and dual-branch contrastive learning (SPD-IRSCL). This model investigates a novel framework that integrates a pre-training module through contrastive learning and introduces a target-aware image reinforcement strategy to enhance predictive performance. Specifically, we employ a dual-encoder network to generate positive and negative sample pairs, enabling the extraction and comparison of fine-grained feature representations for constructing an optimal pre-trained model. Subsequently, we propose an image reinforcement strategy that augments the original datasets by strategically mosaicking targets (polyps) into diverse background images, thus enriching the distribution of training data. Building upon the optimized pre-trained model, we reconstruct a hierarchical detection framework comprising Backbone, Neck, and Head modules. The Backbone module leverages convolutional neural networks to extract multiscale feature representations, while the Neck module employs a primary-auxiliary dual-branch network to facilitate multiscale feature fusion, enabling the integration of different-level semantic information. The Head module utilizes task-specific subnetworks for precise object localization and classification. We verify the SPD-IRSCL model through model comparison, ablation studies, and sensitivity tests, demonstrating superior performance in complex detection scenarios, especially in small-scale medical datasets. The framework synergistically integrates contrastive learning, data reinforcement, and hierarchical feature extraction, showing potential for clinical applications like computer-aided diagnosis, screening, and prevention of cancer, and telehealth services.

Deep learning-based image stitching methods have achieved promising performance on conventional stitching datasets. However, real-world scenarios may introduce challenges such as complex weather conditions, illumination variations, and dynamic scene motion, which severely degrade image quality and lead to significant misalignment in stitching results. To solve this problem, we propose an adverse condition-tolerant image stitching network, dubbed ACDIS. We first introduce a bidirectional consistency learning framework, which ensures reliable alignment through an iterative optimization paradigm that integrates differentiable image restoration and Gaussian-distribute encoded homography estimation. Subsequently, we incorporate motion constraints into the seamless composition network to produce robust stitching results without interference from moving scenes. We further propose the first adverse scene image stitching dataset, which covers diverse parallax and scenes under low-light, haze, and underwater environments. Extensive experiments show that the proposed method can generate visually pleasing stitched images under adverse conditions, outperforming state-of-the-art methods.

Recently, state space models (SSM), particularly Mamba, have attracted significant attention from scholars due to their ability to effectively balance computational efficiency and performance. However, most existing visual Mamba methods flatten images into 1D sequences using predefined scan orders, which results the model being less capable of utilizing the spatial structural information of the image during the feature extraction process. To address this issue, we proposed a novel visual foundation model called DefMamba. This model includes a multi-scale backbone structure and deformable mamba (DM) blocks, which dynamically adjust the scanning path to prioritize important information, thus enhancing the capture and processing of relevant input features. By combining a deformable scanning(DS) strategy, this model significantly improves its ability to learn image structures and detects changes in object details. Numerous experiments have shown that DefMamba achieves state-of-the-art performance in various visual tasks, including image classification, object detection, instance segmentation, and semantic segmentation. The code is open source on DefMamba.

Networked 360(degrees) video has become increasingly popular. Despite the immersive experience for users, its sheer data volume, even with the latest H.266 coding and viewport adaptation, remains a significant challenge to today's networks. Recent studies have shown that integrating deep learning into video coding can significantly enhance compression efficiency, providing new opportunities for high-quality video streaming. In this work, we conduct a comprehensive analysis of the potential and issues in applying neural codecs to 360(degrees) video streaming. We accordingly present NETA, a synergistic streaming scheme that merges neural compression with traditional coding techniques, seamlessly implemented within an edge intelligence framework. To address the non-trivial challenges in the short viewport prediction window and time-varying viewing directions, we propose implicit-explicit buffer-based prefetching grounded in content visual saliency and bitrate adaptation with smart model switching around viewports. A novel Lyapunov-guided deep reinforcement learning algorithm is developed to maximize user experience and ensure long-term system stability. We further discuss the concerns towards practical development and deployment and have built a working prototype that verifies NETA's excellent performance. For instance, it achieves a 27% increment in viewing quality, a 90% reduction in rebuffering time, and a 64% decrease in quality variation on average, compared to state-of-the-art approaches.

Audio-visual segmentation (AVS) plays a critical role in multimodal machine learning by effectively integrating audio and visual cues to precisely segment objects or regions within visual scenes. Recent AVS methods have demonstrated significant improvements. However, they overlook the inherent frequency-domain contradictions between audio and visual modalities–the pervasively interfering noise in audio high-frequency signals vs. the structurally rich details in visual high-frequency signals. Ignoring these differences can result in suboptimal performance. In this paper, we rethink the AVS task from a deeper perspective by reformulating AVS task as a frequency-domain decomposition and recomposition problem. To this end, we introduce a novel Frequency-Aware Audio-Visual Segmentation (FAVS) framework consisting of two key modules: Frequency-Domain Enhanced Decomposer (FDED) module and Synergistic Cross-Modal Consistency (SCMC) module. FDED module employs a residual-based iterative frequency decomposition to discriminate modality-specific semantics and structural features, and SCMC module leverages a mixture-of-experts architecture to reinforce semantic consistency and modality-specific feature preservation through dynamic expert routing. Extensive experiments demonstrate that our FAVS framework achieves state-of-the-art performance on three benchmark datasets, and abundant qualitative visualizations further verify the effectiveness of the proposed FDED and SCMC modules. The code will be released as open source upon acceptance of the paper.