Domain generalization addresses domain shift in real-world applications. Most methods adopt a domain angle, seeking invariant representation across domains by aligning their marginal distributions, irrespective of individual classes, naturally leading to insufficient exploration of discriminative information. Switching to a class angle, we find that multiple domain-related peaks or clusters within the same individual classes must emerge due to distribution shift.In other words, marginal alignment does not guarantee conditional alignment, leading to suboptimal generalization. Therefore, we argue that acquiring discriminative generalization between classes within domains is crucial. In contrast to seeking distribution alignment, we endeavor to safeguard domain-related between-class discrimination. To this end, we devise a novel Conjugate Consistent Enhanced Module, namely Con2EM, based on a distribution over domains, i.e., a meta-distribution. Specifically, we employ a novel distribution-level Universum strategy to generate supplementary diverse domain-related class-conditional distributions, thereby enhancing generalization. This allows us to resample from these generated distributions to provide feedback to the instance-level classifier, further improving its adaptability to the target-agnostic.To ensure generation accuracy, we establish an additional distribution-level classifier to regularize these conditional distributions. Extensive experiments demonstrate its effectiveness and low computational cost compared to recent strong competitors.

Domain Generalization (DG) aims to learn a well-generalized model for unseen target domains from multiple observed source domains. Most current approaches, e.g., domain-invariant representation, typically focus on learning a universal sample-to-label mapping (function) across domains and have achieved impressive performance. Nonetheless, they usually overlook semantically intrinsic domain-specific information, resulting in limited generalizability. For compensation, in this paper, we adopt a novel standpoint, that is, a domain can be regarded as a meta-sample sampling from a certain meta-distribution, namely an environment distribution. With this perspective, in DG, functions learned from individual domains can be seen as a collective set of functional samples. Consequently, we can establish a meta-function, mapping from the environment to functions, to induce specific functions for unseen domains from the above function set effectively. To achieve this meta-function, we propose a learning paradigm based on Gaussian process with theoretical guarantee, namely Generalization Process for Domain Generalization (GPDG). Specifically, analogous to traditional Gaussian process, we describe the inference process in DG as Gaussian process fed with samples and their corresponding domain distributions. Furthermore, we employ a domain augmentation strategy to refine its smoothness. Extensive experiments are constructed to demonstrate the effectiveness of GPDG.

Long-tail recognition remains challenging for pre-trained foundation models like CLIP, which often suffer from performance degradation under imbalanced data. This stems not only from the overfitting/underfitting issues during fine-tuning but, more fundamentally, from the inherent bias inherited from the long-tail distribution of their massive pre-training datasets. To address this, we propose HGLTR (Hierarchy-Guided Long-Tail Recognition), a novel framework that calibrates pre-trained models by injecting objective class hierarchy knowledge. We argue that the semantic proximity defined by a hierarchy provides a robust, data-independent prior to counteract model bias. Our method is specifically designed for vision-language models' dual-modality architecture. At the feature level, we align image embeddings with a hierarchy-guided text similarity structure. At the classifier level, we employ a distillation loss to regularize predictions using soft labels derived from the hierarchy. This dual-level injection effectively transfers knowledge from head to tail classes. Experiments on ImageNet-LT, Places-LT, and iNaturalist 2018 demonstrate that HGLTR achieves state-of-the-art performance, particularly in tail-classes accuracy, highlighting the importance of leveraging structural priors to calibrate foundation models for real-world data.

Matrix-structured parameters frequently appear in many artificial intelligence models such as large language models. More recently, an efficient Muon optimizer is designed for matrix parameters of large-scale models, and shows markedly faster convergence than the vector-wise algorithms. Although some works have begun to study convergence properties (i.e., optimization error) of the Muon optimizer, its generalization properties (i.e., generalization error) is still not established. Thus, in this paper, we study generalization error of the Muon optimizer based on algorithmic stability and mathematical induction, and prove that the Muon has a generalization error of O(1/Nκ^T), where N is training sample size, and T denotes iteration number, and κ>0 denotes minimum difference between singular values of gradient estimate. To enhance generalization of the Muon, we propose an effective mixed Muon (MiMuon) optimizer by cautiously using orthogonalization of gradient, which is a hybrid of Muon and momentum-based SGD optimizers. Then we prove that our MiMuon optimizer has a lower generalization error of O(1/N) than O(1/Nκ^T) of Muon optimizer, since κ generally is very small. Meanwhile, we also studied the convergence properties of our MiMuon algorithm, and prove that our MiMuon algorithm has the same convergence rate of O(1/T^1/4) as the Muon algorithm. Some numerical experimental results on training large models including Qwen3-0.6B and YOLO26m demonstrate efficiency of the MiMuon optimizer.

Adam and AdamW are a class of default optimizers for training deep learning models in machine learning. These adaptive algorithms converge faster but generalize worse compared to SGD. In fact, their proved generalization error O(1/√(N)) also is larger than O(1/N) of SGD, where N denotes training sample size. Recently, although some variants of Adam have been proposed to improve its generalization, their improved generalizations are still unexplored in theory. To fill this gap, in the paper, we restudy generalization of Adam and AdamW via algorithmic stability, and first prove that Adam and AdamW without square-root (i.e., Adam(W)-srf) have a generalization error O(^-2T/N), where T denotes iteration number and >0 denotes the smallest element of second-order momentum plus a small positive number. To improve generalization, we propose a class of efficient clever Adam (i.e., HomeAdam(W)) algorithms via sometimes returning momentum-based SGD. Moreover, we prove that our HomeAdam(W) have a smaller generalization error O(1/N) than O(^-2T/N) of Adam(W)-srf, since is generally very small. In particular, it is also smaller than the existing O(1/√(N)) of Adam(W). Meanwhile, we prove our HomeAdam(W) have a faster convergence rate of O(1/T^1/4) than O(^-1/T^1/4) of the Adam(W)-srf, where ≤ also is very small. Extensive numerical experiments demonstrate efficiency of our HomeAdam(W) algorithms.

The open set known class bias is conventionally viewed as a fatal problem i.e., the models trained solely on known classes tend to fit unknown classes to known classes with high confidence in inference. Thus existing methods, without exception make a choice in two manners: most methods opt for eliminating the known class bias as much as possible with tireless efforts, while others circumvent the known class bias by employing a reconstruction method. However, in this paper, we challenge the two widely accepted approaches and present a novel proposition: the so-called harmful known class bias for most methods is, exactly conversely, beneficial for the reconstruction-based method and thus such known class bias can serve as a positive-incentive to the Open set recognition (OSR) models from a reconstruction perspective. Along this line, we propose the Bias Enhanced Reconstruction Learning (BERL) framework to enhance the known class bias respectively from the class level, model level and sample level. Specifically, at the class level, a specific representation is constructed in a supervised contrastive manner to avoid overgeneralization, while a diffusion model is employed by injecting the class prior to guide the biased reconstruction at the model level. Additionally, we leverage the advantages of the diffusion model to design a self-adaptive strategy, enabling effective sample-level biased sampling based on the information bottleneck theory. Experiments on various benchmarks demonstrate the effectiveness and performance superiority of the proposed method.

Open-Set Domain Generalization (OSDG) aims to generalize over unseen target domains containing open classes, and the core challenge lies in identifying unknown samples never encountered during training. Recently, CLIP has exhibited impressive performance in OSDG, while it still falls into the dilemma between structural risk of known classes and open space risk from unknown classes, and easily suffers from over-confidence, especially when distinguishing known-like unknown samples. To this end, we propose a Semantic-enhanced CLIP (SeeCLIP) framework that leverages fine-grained semantics to boost unknown detection, so as to accommodate both risks and enable precise discrimination among categories. In SeeCLIP, we propose a semantic-aware prompt enhancement module to extract fine-grained key semantic features, and establish a fine-grained vision-language alignment. Duplex contrastive learning is proposed for prompt learning, which jointly optimizes duplex losses such that the unknown prompt is similar to known prompts, yet exhibits key semantic differences. We also design a semantic-guided diffusion module to enable nuanced capture in generation. By injecting perturbed key semantics into a diffusion model as control conditions, it generates the closest unknowns or pseudo-open samples with high similarity yet low belongingness to known classes. We formulate a generalization bound for OSDG, and show that SeeCLIP can achieve a lower generalization risk. Extensive experiments on benchmark datasets validate the superiority of SeeCLIP, it outperforms the SOTA methods by nearly 3% on accuracy and 5% on H-index, respectively.

Irregularly sampled time series (ISTS), characterized by non-uniform time intervals with natural missingness, are prevalent in real-world applications. Existing approaches for ISTS modeling primarily rely on observed values to impute unobserved ones or infer latent dynamics. However, these methods overlook a critical source of learning signal: the reconstruction error inherently produced during model training. Such error implicitly reflects how well a model captures the underlying data structure and can serve as an informative proxy for unobserved values. To exploit this insight, we propose iTimER, a simple yet effective self-supervised pre-training framework for ISTS representation learning. iTimER models the distribution of reconstruction errors over observed values and generates pseudo-observations for unobserved timestamps through a mixup strategy between sampled errors and the last available observations. This transforms unobserved timestamps into noise-aware training targets, enabling meaningful reconstruction signals. A Wasserstein metric aligns reconstruction error distributions between observed and pseudo-observed regions, while a contrastive learning objective enhances the discriminability of learned representations. Extensive experiments on classification, interpolation, and forecasting tasks demonstrate that iTimER consistently outperforms state-of-the-art methods under the ISTS setting.

Imbalanced Domain Generalization (IDG) focuses on mitigating both , both of which fundamentally shape the model's decision boundaries, particularly under heterogeneous long-tailed distributions across domains. Despite its practical significance, it remains underexplored, primarily due to the complexity of handling their entanglement and the paucity of foundations. In this paper, we begin by establishing the generalization bound for IDG, highlighting the role of posterior discrepancy and decision margin. This bound motivates us to focus on directly steering decision boundaries, marking a clear departure from existing methods. Then, we propose a novel Negative-Dominant Contrastive Learning (NDCL) for IDG to enhance discriminability while enforce posterior consistency across domains. Specifically, inter-class decision-boundary separation is enhanced by placing greater emphasis on negatives as the primary signal in our contrastive learning, naturally amplifying gradient signals for minority classes to avoid the decision boundary being biased toward majority classes. Intra-class compactness is encouraged through a re-weighted cross-entropy strategy, and posterior consistency across domains is enforced through a prediction-central alignment strategy. Finally, rigorous yet challenging experiments on benchmarks validate the effectiveness of our NDCL.

Domain Generalization (DG) learns a model from multiple source domains to combat individual domain differences and ensure generalization to unseen domains. Most existing methods focus on learning domain-invariant absolute representations. However, we empirically observe that such representations often suffer from notable distribution divergence, leading to unstable performance in diverse unseen domains. In contrast, relative representations, constructed w.r.t. a set of anchors, naturally capture geometric relationships and exhibit intrinsic stability within a dataset. Despite this potential, their application to DG remains largely unexplored, due to their common transductive assumption that anchors require access to target-domain data, which is incompatible with the inductive setting of DG. To address this issue, we design Re2SL, a simple and lightweight plug-in module that follows a pre-trained encoder and constructs anchors solely from source-domain prototypes, thereby ensuring a completely inductive design. To our knowledge, Re2SL is the first to explore relative representation for DG. This design is inspired by the insight that ReSidual differences between absolute and domain-specific representations can spontaneously seek stable representations within the same distribution shared across all domains. Leveraging these stable representations, we construct cross-domain ReLative representation to enhance stability and transferability without accessing any target data during training or anchor computation. Empirical studies show that our constructed representation exhibits minimal H-divergence, confirming its stability. Notably, Re2SL achieves up to 4.3% improvement while reducing computational cost by 90%, demonstrating its efficiency.

Real-world datasets often follow a long-tailed distribution, making generalization to tail classes difficult. We revisit this problem through the lens of shortcut learning, where models prefer the easiest predictive cues (e.g., background or textures) over object-centric semantics, especially under scarce and biased supervision. We find that this tendency is amplified for tail classes: limited examples often share similar contexts, making non-semantic signals highly correlated and thus tempting shortcuts, whereas head classes with diverse appearances and environments encourage more stable object-focused representations. Motivated by this observation, we propose Shortcut-Resistant CAM Distillation (SRCD), a plug-and-play framework that transfers object-focused explanations from head to tail classes. SRCD operates in the Class Activation Map (CAM) space, where a CAM provides a class-specific spatial evidence map for a prediction. SRCD aggregates CAMs from a small set of head-class candidates into a shortcut-resistant teacher using an energy-model weighting based on coherence (Laplacian smoothness) and concentration (Hoyer sparsity), and distills it to the tail-class CAM. We provide a theoretical analysis that quantifies shortcut reliance as shortcut-region evidence mass in CAM space and shows that SRCD suppresses tail shortcuts. Extensive experiments on long-tailed benchmarks consistently improve strong baselines.

Lion optimizer is a popular learning-based optimization algorithm in machine learning, which shows impressive performance in training many deep learning models. Although convergence property of the Lion optimizer has been studied, its generalization analysis is still missing. To fill this gap, we study generalization property of the Lion via algorithmic stability based on the mathematical induction. Specifically, we prove that the Lion has a generalization error of O(1/Nτ^T), where N is training sample size, and τ>0 denotes the smallest absolute value of non-zero element in gradient estimator, and T is the total iteration number. In addition, we obtain an interesting byproduct that the SignSGD algorithm has the same generalization error as the Lion. To enhance generalization of the Lion, we design a novel efficient Cautious Lion (i.e., CLion) optimizer by cautiously using sign function. Moreover, we prove that our CLion has a lower generalization error of O(1/N) than O(1/Nτ^T) of the Lion, since the parameter τ generally is very small. Meanwhile, we study convergence property of our CLion optimizer, and prove that our CLion has a fast convergence rate of O(√(d)/T^1/4) under ℓ_1-norm of gradient for nonconvex stochastic optimization, where d denotes the model dimension. Extensive numerical experiments demonstrate effectiveness of our CLion optimizer.

Recent studies have demonstrated that semi-supervised learning (SSL) is highly vulnerable to backdoor attacks, where adversaries can manipulate up to 90% of model predictions through just a tiny fraction of poisoned training data. Despite the widespread adoption of SSL in safety-critical applications, effective defenses against such attacks remain limited. In this paper, we present a comprehensive defense framework designed to protect SSL against sophisticated backdoor attacks. Our work begins with a systematic analysis of backdoor mechanisms in SSL from two critical perspectives: (1) how attackers establish persistent correlations between triggers and target classes; (2) how triggers are introduced and resist removal at the data level. Our investigation reveals that, unlike supervised learning, SSL backdoor attacks (1) uniquely exploit pseudo-labeling mechanisms to establish stronger trigger-target correlations, and (2) demonstrate remarkable resilience at the data level, with triggers potentially appearing in any frequency band (low, medium, or high). Based on these insights, we introduce Backdoor Invalidator (BI), a defense framework that integrates three novel techniques: complementary learning, trigger mix-up, and dual domain filtering, which collectively obstruct, dilute, and filter the influence of backdoor attacks in both feature learning and data processing. Through extensive evaluation against state-of-the-art attacks, BI significantly reduces the average attack success rate while maintaining comparable accuracy on clean data. We also provide theoretical guarantees for BI’s generalization capability and demonstrate its practical deployability as a plug-in component. The code of this work is available at https://github.com/wxr99/Backdoor Invalidator4SSL.

Large models recently are widely applied in artificial intelligence, so efficient training of large models has received widespread attention. More recently, a useful Muon optimizer is specifically designed for matrix-structured parameters of large models. Although some works have begun to studying Muon optimizer, the existing Muon and its variants still suffer from high sample complexity or high memory for large models. To fill this gap, we propose a light and fast Muon (LiMuon) optimizer for training large models, which builds on the momentum-based variance reduced technique and randomized Singular Value Decomposition (SVD). Our LiMuon optimizer has a lower memory than the current Muon and its variants. Moreover, we prove that our LiMuon has a lower sample complexity of O(ε^-3) for finding an ε-stationary solution of non-convex stochastic optimization under the smooth condition. Recently, the existing convergence analysis of Muon optimizer mainly relies on the strict Lipschitz smooth assumption, while some artificial intelligence tasks such as training large language models (LLMs) do not satisfy this condition. We also proved that our LiMuon optimizer has a sample complexity of O(ε^-3) under the generalized smooth condition. Numerical experimental results on training DistilGPT2 and ViT models verify efficiency of our LiMuon optimizer.

Irregularly sampled multivariate time series (ISMTS) are prevalent in reality. Most existing methods treat ISMTS as synchronized regularly sampled time series with missing values, neglecting that the irregularities are primarily attributed to variations in sampling rates. In this paper, we introduce a novel perspective that irregularity is essentially relative in some sense. With sampling rates artificially determined from low to high, an irregularly sampled time series can be transformed into a hierarchical set of relatively regular time series from coarse to fine. We observe that additional coarse-grained, relatively regular time series not only mitigate the irregularly sampled challenges but also incorporate broad-view temporal information, thereby serving as a valuable asset for representation learning. Therefore, following the philosophy of learning that sees the big picture first, then delving into the details, we present the multi-scale and multi-correlation attention network (MuSiCNet), combining multiple scales to iteratively refine the ISMTS representation. Specifically, within each scale, we explore time attention and frequency correlation matrices to aggregate intra- and inter-series information, naturally enhancing the representation quality with richer and more intrinsic details. Across adjacent scales, we employ a representation rectification method containing contrastive learning and reconstruction results adjustment to further improve representation consistency. Experimental results demonstrate that MuSiCNet consistently achieves competitive performance with state-of-the-art methods across four key ISMTS tasks: classification, interpolation, forecasting, and anomaly detection.

Online Continual Learning (OCL) aims to learn from endless non\text{-}stationary data streams, yet most existing methods assume a flat label space and overlook the hierarchical organization of real\text{-}world concepts that evolves both horizontally (sibling classes) and vertically (coarse or fine categories). To better reflect this context, we introduce a new problem setting, DHOCL (Online Continual Learning from Dynamic Hierarchies), where taxonomies evolve across granularities and each sample provides supervision at a single hierarchical level. In this setting, we find two fundamental issues: (i) partial supervision under mixed granularities provides only point-wise signals over an evolving path-wise hierarchy, which constrains plasticity and undermines cross-level semantic consistency, and (ii) the dynamically evolving hierarchies induce granularity-dependent interference, destabilizing popular replay and regularization mechanisms and thereby exacerbating catastrophic forgetting. To tackle these issues, we propose HALO (Hierarchical Adaptive Learning with Organized Prototypes), which adaptively combines complementary classification heads, regularized by organized learnable hierarchical prototypes, enabling rapid adaptation, hierarchical consistency, and structured knowledge consolidation as the taxonomy evolves. Extensive experiments on multiple benchmarks demonstrate that HALO consistently outperforms existing methods across hierarchical accuracy, mistake severity, and continual performance.

Recent advances in leveraging pre-trained Vision-Language Models (VLMs), such as CLIP, for open-set recognition (OSR) have mainly focused on learning textual prompts for known classes. Despite promising performance, textual-prompt-based methods still suffer from three limitations: insufficient characterization of low-level visual attributes, interference from image backgrounds, and distribution discrepancy between VLM pre-training data and target-task data. To address these issues, we develop a \underline{T}arget-\underline{A}ware \underline{D}ual \underline{P}rompt \underline{L}earning framework (TADPL). Building upon visual-prototype prompting as a complementary representation to textual prompts, TADPL introduces a new Target-Aware Enhancement (TAE) module that identifies target-aware known samples and constructs hard pseudo-unknown samples in the vicinity of known classes, thereby suppressing background interference and tightening known-class decision boundaries. A data-adaptive mechanism is further incorporated to improve the adaptability of the pre-trained VLM to the target data distribution. Moreover, we theoretically characterize the fusion gain induced by textual–visual prompt complementarity and establish a transfer bound relating pseudo-unknown separation to real-unknown separation. Extensive experiments on four standard OSR benchmarks demonstrate the effectiveness of TADPL, achieving average AUROC and OSCR scores of 97.4\% and 95.4\%, respectively, and consistently outperforming strong existing baselines.

Existing learning-based visual retouching primarily focuses on improving image quality through end-to-end objective mapping between input and retouched images. However, these approaches often overlook two critical aspects: the progressive nature of image retouching and the subjective aesthetic preferences, resulting in suboptimal visual outcomes. To address this, we introduce Automatic Aesthetic Image Retouching via active reinforcement learning (A$^{3}$3RL) to enhance the visualization experience in two sub-tasks: color enhancement and composition optimization, which are formulated as a unified Markov Decision Process in the proposed A$^{3}$3RL framework. In our approach, each pixel functions as an autonomous agent that determines optimal actions based on aesthetic guidance, engaging in online exploration through immediate pixel-wise and channel-wise feedback from the aesthetic environment. By leveraging a pretrained image aesthetic model, our method ensures that the A$^{3}$3RL process aligns with human aesthetic preferences and adheres to subjective aesthetic principles. The framework integrates pixel-level retouching actions with image-level operations to achieve optimal image sequences through progressive iterations. Extensive experiments demonstrate that our method effectively recalibrates image aesthetics across multiple dimensions: low-level quality metrics (PSNR, SSIM), visual perception (LPIPS), and subjective visual experience (human survey). The results demonstrate high consistency with expert-retouched ground-truth images.

The integration of Large Language Models (LLMs) into e-commerce platforms has significantly enhanced user experience through personalized recommendations and automated customer support. However, existing Retrieval-Augmented Generation (RAG) frameworks face challenges when applied to e-commerce product Question Answering (QA), such as handling extensive product catalogs, ensuring timely knowledge updates, and maintaining efficient retrieval performance. In this paper, we propose ItemRAG, a novel framework that combines RAG with item-based knowledge computing to address these challenges. ItemRAG decouples QA templates from specific products by leveraging a dynamic knowledge graph, enabling efficient updates and reducing the size of the knowledge base. The framework includes state analysis to capture user intent and context, grouped indexing for efficient retrieval, and knowledge computing to dynamically generate accurate answers. Experimental results demonstrate that decoupled-based ItemRAG significantly outperforms the Coupled-based RAG approaches (CoupledRAG) in retrieval accuracy and generation quality, achieving higher precision, recall, F1-score, and factual correctness. Our work highlights the efficacy of integrating the knowledge graph with RAG to enhance LLM-based e-commerce customer service systems.