Unsupervised incremental class discovery (UICD) is a novel scenario in which a model must continuously learn new categories from a stream of unlabeled data while retaining previously acquired knowledge. Existing methods primarily leverage the representation capabilities of pre-trained models (PTMs) but fail to explore underlying feature spaces and neglect data relationships, limiting the PTMs' effectiveness. To address these challenges, we propose the self-distillation reciprocity knowledge discovery (SDRKD) framework to unlock the significant PTMs' potential. Specifically, we introduce the Reciprocity Nearest-Neighbor Feature Fusion (R2NF) method to generate reciprocally structured samples, optimizing the updates to the teacher-student self-distillation network by maximizing mutual information, thereby obtaining a more discriminative model for the novel class. Moreover, we propose a reciprocity classifier calibration (R2C) method, and utilize the correlation between classifiers to calibrate the new classifiers, reducing the overlap in the feature space during the cascading process, thus minimizing the forgetting of previously learned knowledge. Furthermore, our method reduces memory loss of previously learned knowledge while enhancing the model's ability to discover new classes. Extensive experiments demonstrate that our method achieves state-of-the-art performance on various benchmarks.
Video inpainting aims to restore missing regions while preserving spatial and temporal coherence. Diffusion-based methods achieve strong per-frame reconstruction, but their sampling implicitly generates temporally coupled latent trajectories whose long-horizon stability is not explicitly modeled, leading to a trade-off between temporal consistency and structural detail. We revisit video inpainting from the perspective of temporal trajectory stability, viewing temporal inconsistency as instability along time-indexed denoising trajectories rather than an output-level error. Based on this view, we propose an inference-time trajectory stabilization framework that monitors motion-aligned deviation and triggers risk-aware correction only when instability accumulates. It combines sparsely sampled trajectory anchors as stability references with neighborhood-consistent propagation to regulate trajectory evolution while preserving local generative freedom. Implemented as a lightweight control layer in the sampling loop, it selectively contracts unstable trajectories toward motion-consistent manifolds instead of enforcing uniform temporal constraints. Experiments show consistent improvements in temporal coherence and structural fidelity.
Multimodal fusion focuses on integrating information from multiple modalities with the goal of more accurate prediction, which has achieved remarkable progress in a wide range of scenarios, including autonomous driving and medical diagnosis. However, the reliability of multimodal fusion remains largely unexplored especially under low-quality data settings. This paper surveys the common challenges and recent advances of multimodal fusion in the wild and presents them in a comprehensive taxonomy. From a data-centric view, we identify four main challenges that are faced by multimodal fusion on low-quality data, namely (1) noisy multimodal data that are contaminated with heterogeneous noises, (2) incomplete multimodal data that some modalities are missing, (3) imbalanced multimodal data that the qualities or properties of different modalities are significantly different and (4) quality-varying multimodal data that the quality of each modality dynamically changes with respect to different samples. This new taxonomy will enable researchers to understand the state of the field and identify several potential directions. We also provide discussion for the open problems in this field together with interesting future research directions.
Video generation is pivotal to digital media creation, and recent advances in autoregressive video generation have markedly enhanced the efficiency of real-time video synthesis. However, existing approaches generally rely on heuristic KV Cache policies, which ignore differences in token importance in long-term video generation. This leads to the loss of critical spatiotemporal information and the accumulation of redundant, invalid cache, thereby degrading video generation quality and efficiency. To address this limitation, we first observe that token contributions to video generation are highly time-heterogeneous and accordingly propose a novel Past- and Future-Informed KV Cache Policy (PaFu-KV). Specifically, PaFu-KV introduces a lightweight Salience Estimation Head distilled from a bidirectional teacher to estimate salience scores, allowing the KV cache to retain informative tokens while discarding less relevant ones. This policy yields a better quality-efficiency trade-off by shrinking KV cache capacity and reducing memory footprint at inference time. Extensive experiments on benchmarks demonstrate that our method preserves high-fidelity video generation quality while enables accelerated inference, thereby enabling more efficient long-horizon video generation. Our code will be released upon paper acceptance.
Non-negative matrix factorization (NMF) is a powerful technique for clustering analysis. Applying it to multi-view clustering, the existing methods typically apply NMF to obtain soft label matrices for each view independently and then fuse the obtained label matrices of views to generate view-consistent label matrix. A major drawback is that they mainly focus on decision-level fusion and can't effectively exploit the implicit relationships between views at the data level, which are critical for multi-view clustering. Moreover, they rely on the relationship between NMF and K-means to derive labels, which has weak interpretability. To address these problems, we propose a novel multi-view clustering model based on non-negative tensor factorization (NTF), which employs multi-level fusion (both data-level and decision-level) to achieve view-consistent labels for multi-view data. Specifically, we present a non-negative tensor factorization and utilize it to decompose tensorized anchor graph into the product of an anchor indicator tensor and a data indicator tensor from a probabilistically interpretable perspective, thereby enhancing the interpretability of our proposed model. Extensive experimental results demonstrate the effectiveness of our method.
Adaptive navigation in unfamiliar indoor environments is crucial for household service robots. Despite advances in zero-shot perception and reasoning from vision-language models, existing navigation systems still rely on single-pass scoring at the decision layer, leading to overconfident long-horizon errors and redundant exploration. To tackle these problems, we propose Dual-Stance Cooperative Debate Navigation (DSCD-Nav), a decision mechanism that replaces one-shot scoring with stance-based cross-checking and evidence-aware arbitration to improve action reliability under partial observability. Specifically, given the same observation and candidate action set, we explicitly construct two stances by conditioning the evaluation on diverse and complementary objectives: a Task-Scene Understanding (TSU) stance that prioritizes goal progress from scene-layout cues, and a Safety-Information Balancing (SIB) stance that emphasizes risk and information value. The stances conduct a cooperative debate and make policy by cross-checking their top candidates with cue-grounded arguments. Then, a Navigation Consensus Arbitration (NCA) agent is employed to consolidate both sides' reasons and evidence, optionally triggering lightweight micro-probing to verify uncertain choices, preserving NCA's primary intent while disambiguating. Experiments on HM3Dv1, HM3Dv2, and MP3D demonstrate consistent improvements in success and path efficiency while reducing exploration redundancy.
Long-horizon robot policies require compact access to recent observations and reusable experience without expanding the vision-language-action (VLA) context. We introduce Remember Smarter (RS), a plug-and-play module with complementary visual-history and hyperbolic experience-memory branches. Its visual branch compresses multi-view patch histories using bidirectional spatial Mamba and causal temporal Mamba, then exposes the resulting memory to action-facing hidden states through residual cross-attention while leaving the VLM visual-token stream unchanged. Its experience branch stores successful final-layer VLM states in a Poincare VAE space, organizes them hierarchically, and asynchronously converts retrieved experience into geodesic prompt tokens without blocking action inference. When adapted to pi0, RS increases total success on LIBERO-Plus from 53.6 achieves substantial performance gains in real-robot experiments designed to evaluate memory retention and experience utilization.
Recent advances in video generation have produced models capable of synthesizing stunning visual content from simple text prompts. However, these models struggle to generate long-form, coherent narratives from high-level concepts like dialogue, revealing a ``semantic gap'' between a creative idea and its cinematic execution. To bridge this gap, we introduce a novel, end-to-end agentic framework for dialogue-to-cinematic-video generation. Central to our framework is ScripterAgent, a model trained to translate coarse dialogue into a fine-grained, executable cinematic script. To enable this, we construct ScriptBench, a new large-scale benchmark with rich multimodal context, annotated via an expert-guided pipeline. The generated script then guides DirectorAgent, which orchestrates state-of-the-art video models using a cross-scene continuous generation strategy to ensure long-horizon coherence. Our comprehensive evaluation, featuring an AI-powered CriticAgent and a new Visual-Script Alignment (VSA) metric, shows our framework significantly improves script faithfulness and temporal fidelity across all tested video models. Furthermore, our analysis uncovers a crucial trade-off in current SOTA models between visual spectacle and strict script adherence, providing valuable insights for the future of automated filmmaking.
Large language models (LLMs) store extensive factual knowledge acquired during pretraining, yet this knowledge is inherently static and may become inaccurate or outdated, leading to knowledge hallucinations. Knowledge editing offers an efficient alternative to full retraining by enabling targeted factual updates while preserving overall model behavior. Existing locate-then-edit methods, however, rely on fixed layer selection strategies, treating the locating stage as a static design choice and failing to account for the hierarchical and instance-dependent nature of knowledge representation in LLMs. In this paper, we propose FiDAL, a Fisher-driven adaptation-aware locating strategy that dynamically identifies which model components should be edited for a given knowledge update. FiDAL formulates localization as a weight-level decision problem and leverages Fisher Information to select layers that are both influential and sensitive to factual modifications. A lightweight probing stage with low-rank modulation enables efficient localization with minimal overhead. Experiments on standard benchmarks demonstrate that FiDAL consistently improves editing effectiveness and knowledge preservation across multiple editing methods.
This paper pays attention to open-vocabulary 3D object affordance grounding (OVAG), which aims to localize affordance regions on 3D objects by leveraging interaction images or textual instructions. Most existing methods treat interaction images as sources of external affordance knowledge and align them with 3D visual representations, while overlooking the intrinsic relationship between local object attributes and affordances, which limits localization accuracy and generalization. For instance, a cup handle affords grasping due to its curved shape and appropriate thickness, indicating that affordances emerge from specific attribute compositions rather than global object appearance. Motivated by this, we propose Attribute-Affordance Hierarchies (AAH) learning framework that explicitly models the hierarchical relationships between object-region attributes and affordances. Our approach first captures local region relationships using hypergraph, and then projects these region-level concepts into a hyperbolic space to encode their hierarchical organization. Furthermore, we introduce counterfactual attribute samples to encourage robust learning of attribute–affordance dependencies under varying conditions. By jointly modeling visual structure and hierarchical concept information, our method achieves more accurate affordance localization. Extensive experiments and qualitative analyses demonstrate the effectiveness of our approach.
Multi-view clustering based on anchor graphs has attracted significant attention due to its ability to substantially reduce computational complexity, enabling the efficient processing of large-scale multimedia data. However, most existing anchor graph-based clustering methods fail to fully exploit the intrinsic properties of anchor graphs when applying regression techniques. Moreover, some approaches focus solely on sample labels while overlooking the crucial role of anchor labels in clustering. To address these limitations, we leverage the probabilistic information of the anchor graph by employing probabilistic projection to map the anchor graph into the label space, thereby obtaining anchor labels. By clustering both anchors and samples simultaneously, the anchor graph serves as a guide to induce anchor labels, which are then used to generate sample labels, facilitating anchor-guided sample clustering. Furthermore, we propose a novel regularization paradigm based on the matrix nuclear norm, ensuring that the obtained results remain discrete and that the sample distribution across clusters is balanced. Additionally, we introduce a new matrix nuclear norm optimization method based on the first-order Taylor expansion. Extensive experiments on real-world datasets demonstrate the effectiveness and robustness of our proposed method, achieving superior performance compared to state-of-the-art approaches. Our code is available at: https://github.com/harunakai/ADMC
Object Concept Learning (OCL) aims to recognize high-level attributes and affordances of objects and to infer the causal relationships between them. The key is to accurately model the many-to-many mapping between objects and concepts: While an object may possess multiple concepts, a concept can also belong to multiple objects. Existing methods primarily rely on attention mechanisms to capture label correlations, which limits their ability to comprehend high-level concepts and to perform effective causal reasoning. Inspired by the human cognitive process of progressive understanding, a Hierarchical Cross-Modal Relational Reasoning (CORE) framework is proposed to enhance the understanding of object concepts through hierarchical interaction and reasoning between visual and textual modalities. Specifically, a coarse-to-fine relational reasoning module is developed, in which multi-step learnable prompts are employed to progressively localize the conceptual information of objects, thereby improving the accuracy of object-concept mapping. Subsequently, to facilitate the modeling of causal relationships between object attributes and affordances, a counterfactual reasoning mechanism is introduced. By constructing counterfactual samples and distinguishing the predictive outputs of factual and counterfactual parts, the model's ability to capture causality among concepts is enhanced. Significant performance gains and extensive visualization analysis demonstrate the superiority of our method.
Multi-view clustering methods based on anchor graphs leverage anchor graphs to reduce computational complexity and enhance operational efficiency. However, existing methods primarily utilize regression or matrix factorization to obtain sample labels, and these methods have the following limitations: 1. They ignore the probabilistic properties of anchor graph. 2. They focus only on the sample label matrix, and ignore the connection between the anchor label matrix and the sample label matrix. To address these issues, we propose a probabilistic regression-based multi-view clustering with anchor graphs. Specifically, it utilizes the data representation of the anchor graph to obtain both sample labels and anchor labels, and allows the anchor labels to guide the learning of the sample labels. Additionally, We introduce the nuclear norm, maximizing it to achieve a balanced distribution of anchors in clusters, and present an optimization method based on the first-order Taylor expansion of the nuclear norm. Extensive experimental results demonstrate the superiority of our method over existing methods.
Cloud-Edge Continual Test-Time Adaptation (CTTA)—with edge devices processing real-time data and the cloud offering strong computing power—is a critical paradigm for models that adapt to dynamic data distributions in real-world scenarios. However, most existing frameworks assume architectural homogeneity between cloud and edge CNNs, which poses a significant performance bottleneck, particularly given the rapid emergence of Transformer-based models. Current methods fail to bridge this architectural gap, resulting in significant deficiencies in adaptation accuracy and practical applicability. To address this, we propose a novel Cross-Architecture Adaptation (CAA) framework for heterogeneous Cloud-Edge CTTA that enables effective adaptation to shifting data distributions. Specifically, CAA deploys a large Transformer-based teacher model on the cloud for robust feature extraction and prediction, and a lightweight CNN-based student model on edge devices to fit resource constraints. Based on such cloud-edge models, a synergistic edge-to-cloud communication strategy, Multi-criteria Dynamic Cross-domain Sampling, ensures only the most informative, class-balanced samples are uploaded, minimizing communication costs while guaranteeing stable, unbiased adaptation. Moreover, a Multi-level Adaptive Heterogeneous Distillation module is proposed to facilitate effective knowledge transfer across the architecturally disparate models, and improve the learning efficiency of the edge one. Experiments on several benchmarks demonstrate that CAA achieves state-of-the-art performance with low edge resource consumption and minimal edge-to-cloud communication overhead.
Zero-shot learning (ZSL) seeks to empower models with the ability to identify novel classes by transferring knowledge from previously encountered categories. Despite promising progress, most current ZSL methods rely heavily on pre-trained backbone networks for visual feature extraction. These features often contain distracting components that diminish the distinctiveness of critical information, thereby undermining knowledge transfer effectiveness. Moreover, existing methods typically overlook the impact of noisy or irrelevant details embedded in attribute representations. To address these limitations, we propose Discriminative and Noise-Robust Embedding (DNRE) for ZSL. It incorporates two major components: (1) a Channel Covariance Adaptive Enhancement (CCAE) module that captures higher-order dependencies and dynamically emphasizes informative channels to refine visual representations; and (2) a Dynamic Calibration Mechanism (DCM) that improves the alignment between attributes and local visual regions by suppressing noise and irrelevant signals. Comprehensive experiments on three widely-used ZSL benchmarks demonstrate that our method consistently surpasses state-of-the-art baselines, particularly in terms of accurate and robust attribute localization.
The latest progress in text-to-3D generative models makes it possible to generate high-quality 3D content. Recent text-to-3D large model have achieved remarkable breakthroughs in multi-view consistency. However, their effectiveness is often affected by inherent biases, resulting in sensitivity to design settings such as prompt format, leading to difficulty understanding complex prompts. To help text-to-3D generative models understand more diverse prompts, we propose a framework to localize and mitigate the bias in the current text-to-3D large model. Specifically, we first use the existing model to generate 3D content and use the quality evaluation model to identify the cross-modality bias. Then, we use the predicted quality score to quantify the contribution of the prompt text to the bias. Finally, in order to reduce these biases, we construct diverse pairwise examples to help the current text-to-3D large model construct unbiased visual-text connections. The experiment shows that our method has achieved competitive results and can provide higher quality, more diverse 3D content compared to existing methods.
Decoding visual stimuli from electroencephalography (EEG) signals is a crucial step toward practical brain–computer interfaces (BCIs). However, this task requires large-scale and high-quality EEG–image paired datasets. Compared with abundant image data, the limited EEG recordings restrict the decoding models’ performance. To address this challenge, we propose EEGiT, a framework that converts sequential EEG signals into image-like EEG patches and enables the direct use of a pretrained Vision Transformer (ViT) as the EEG encoder. To preserve the spatial topology of brain regions and minimize distributional differences across channels, we group EEG electrodes according to anatomical structures and apply linear interpolation along the spatial dimension. We then resample the EEG signals to align the structure of EEG patches with that of image patches in ViT. This design encourages effective transfer of visual priors learned from large-scale image datasets to EEG representation learning. Experiments on the THINGS-EEG and EEG-3D datasets show that fine-tuning pretrained ViTs improves EEG-to-image retrieval and EEG-based visual classification, while maintaining robustness and strong cross-subject generalization. These results demonstrate a promising direction for leveraging powerful vision models to mitigate data scarcity in EEG decoding.
As Large Language Models (LLMs) become increasingly prevalent, their security vulnerabilities have already drawn attention. Machine unlearning is introduced to seek to mitigate these risks by removing the influence of undesirable data. However, existing methods not only rely on the retained dataset to preserve model utility, but also suffer from cumulative catastrophic utility loss under continuous unlearning requests. To solve this dilemma, we propose a novel method, called Rotation Control Unlearning (RCU), which leverages the rotational salience weight of RCU to quantify and control the unlearning degree in the continuous unlearning process. The skew symmetric loss is designed to construct the existence of the cognitive rotation space, where the changes of rotational angle can simulate the continuous unlearning process. Furthermore, we design an orthogonal rotation axes regularization to enforce mutually perpendicular rotation directions for continuous unlearning requests, effectively minimizing interference and addressing cumulative catastrophic utility loss. Experiments on multiple datasets confirm that our method without retained dataset achieves SOTA performance.
Online Action Detection (OAD) requires real-time prediction of ongoing actions without access to future frames, posing challenges in balancing computational efficiency and long-term dependencies modeling.Existing methods either suffer from slow training and limited temporal receptive fields, or face high computational costs and delayed inference, lacking the capability to tackle extra-long video inputs. Thus, we present a novel Mamba-based OAD framework (MOAD) that efficiently and effectively performs OAD.The hierarchical memory mechanism is introduced to intelligently store high-value scene and action frames based on motion-aware similarity metrics, preserving essential historical knowledge in an online manner. To further reduce storage space, we design a memory quantization method to compress the stored historical features. Additionally, the temporal soft pruning strategy built upon the memory bank is proposed to dynamically remove redundant features, reducing temporal redundancy while maintaining temporal coherence. Sufficient experiments on four challenging benchmarks prove our method significantly outperforms existing methods.