Multi-view multi-label learning frequently suffers from simultaneous feature absence and incomplete annotations, due to challenges in data acquisition and cost-intensive supervision. To tackle the complex yet highly practical problem while overcoming the existing limitations of feature recovery, representation disentanglement, and label semantics modeling, we propose an Adaptive Disentangled Representation Learning method (ADRL). ADRL achieves robust view completion by propagating feature-level affinity across modalities with neighborhood awareness, and reinforces reconstruction effectiveness by leveraging a stochastic masking strategy. Through disseminating category-level association across label distributions, ADRL refines distribution parameters for capturing interdependent label prototypes. Besides, we formulate a mutual-information-based objective to promote consistency among shared representations and suppress information overlap between view-specific representation and other modalities. Theoretically, we derive the tractable bounds to train the dual-channel network. Moreover, ADRL performs prototype-specific feature selection by enabling independent interactions between label embeddings and view representations, accompanied by the generation of pseudo-labels for each category. The structural characteristics of the pseudo-label space are then exploited to guide a discriminative trade-off during view fusion. Finally, extensive experiments on public datasets and real-world applications demonstrate the superior performance of ADRL.

This paper aims to provide error bounds on physics-informed neural network (PINN) in solving Korteweg–de Vries (KdV) equations. We prove that a neural network equipped with two hidden layers and the tanh activation function can reduce the partial differential equation residuals arbitrarily. The generalization error and training error can be bounded by the number of training points and the width of the aforementioned neural network. Besides, the upper bound of the total error can be controlled by the generalization error. These error bounds offer a theoretical understanding of PINN’s ability in solving KdV equations. A series of parameterized KdV equations are also conducted to demonstrate the performance of PINN when solving KdV equations.

Label shift occurs when the conditional distributions remain consistent between source and target domains, but the marginal label distributions differ. For instance, during the early stage of the COVID-19 outbreak, the proportion of pneumonia cases compared to common cold cases in hospitals may have been relatively low. This ratio could shift dramatically in later stages of the pandemic, with pneumonia cases becoming predominant, even though the symptomatic presentation of each disease remained consistent. Existing label shift methods typically aim to adapt a classifier's output to match the target domain's label distribution, assuming the source domain has clean labels. However, real-world scenarios often involve label noise in the source domain. For example, during COVID-19's early phase, mild and confusable symptoms frequently led to misdiagnoses of COVID-19 as the common cold, introducing label noise. Such noise compromises the effectiveness of traditional methods, necessitating novel approaches. To address this, we analyze classifier error bounds under label shift correction using noisy source data. Based on this analysis, we propose a Noise-robust Label Shift Correction (NLSC) framework. NLSC employs a Three-Head Architecture Training (THAT) strategy for robust feature learning and a Class-Adaptive Threshold Cleaning (CATC) strategy for source data purification. Extensive experiments confirm that our method outperforms existing state-of-the-art techniques, particularly in real-world scenarios with high source domain noise rates.

Feature dynamics have emerged as a critical topic about open-environment learning due to the instability of feature availability. While traditional feature evolution targets single-label tasks, multi-label learning is essential to accommodate the exploding annotation spaces. However, multi-label classification with incremental and decremental features is a crucial yet underexplored problem, which poses the challenge of preserving feature representations and label correlations from historical instances and simultaneously adapting to newly arriving streaming data. To address these issues, we propose a two-stage, one-pass learning approach termed MLID. It attempts to compress the informative content of vanished features into the domain of survived ones, facilitate the propagation of label dependencies via low-rank regularization of the classifier, and incorporate augmented features to construct an adaptive classification mechanism. Besides, we design optimization strategies for each stage and provide theoretical guarantees of convergence. Moreover, we establish the generalization error bound of MLID and demonstrate that the compactness of the trace norm and the reuse of models based on effective features can enhance the generalization performance. Finally, we extend it to multi-shot case and extensive experimental results validate the superiority of our MLID.

In real-world applications, data acquisition processes evolve continuously, making dynamic feature learning a prevalent and representative scenario in open environments. As features evolve over time, multi-label classification tasks require the integration of informative knowledge from the original feature space and label space with newly emerging data to support predictions at the current time step. However, the process of information transfer faces two major challenges, which are dynamic capture of label correlations and evolving discriminative semantics of features. To address these challenges, a multi-label classification method with incremental and decremental features named FDCLCP, which integrates feature semantic screening and high-rank modeling of label correlations. Specifically, FDCLCP first employs a weighting mechanism to model the changing contributions of vanished features during knowledge transfer, thereby capturing the varying importance of features in label prediction. Subsequently, we introduce residual learning to inherit essential feature information, while high-rank modeling is leveraged to accommodate evolving label correlations and characterize complex and complete multi-label structures. Moreover, FDCLCP develops tailored optimization strategies for each stage, accompanied by theoretical convergence guarantees. The generalization analysis demonstrates that the quality of previously learned features together with residual learning of newly introduced features contributes to improve the performance. Finally, FDCLCP is extended to multi-shot setting, enabling two types of predictions at arbitrary time steps across both current and previous stages. It is further applied to real-world data for personalized and real-time music recommendation. Extensive experimental results verify the superiority of our FDCLCP.

Multi-label cross-modal retrieval aims to bridge the semantic gap between heterogeneous data by capturing rich co-occurring concepts. Existing methods can be advanced by addressing key aspects such as reducing their dependence on explicit sample correspondences and more fully leveraging multi-label information. Toward these goals, we propose a framework termed Tripartite Similarity Enhanced Canonical Correlation Fusion (TSCCF). TSCCF introduces a generalized correlation matrix to replace the standard empirical covariance matrix, providing a robust foundation for canonical correlation fusion. Built upon this matrix, the framework comprehensively measures cross-modal correspondences by jointly modeling the tripartite interactions among labels, features, and samples. Specifically, a sample-level alignment component leverages overlapping multi-label information as soft-pairing guidance to relax the dependency on explicit pairs and a feature-level interaction component captures the complex nonlinear dependencies between features and co-occurring labels. Comprehensive experiments on multiple benchmark datasets demonstrate that TSCCF shows promising performance across various retrieval tasks, improving retrieval accuracy compared to baseline methods in scenarios where explicit sample pairings are lacking.

Imbalanced kernel clustering, distinguished by differing sample counts among diverse clusters, has gained significant prominence in a multitude of real-world nonlinear data mining scenarios. Nevertheless, the computational requirements of such approaches are often associated with the kernel matrix and display a quadratic increase in relation to the data volume, making it unfeasible for scenarios involving large-scale imbalanced datasets. Moreover, despite the importance of theoretical analysis in machine learning, fast imbalanced kernel clustering methods still lack solid statistical guarantees. Understanding the statistical properties of fast imbalanced kernel clustering therefore remains an important and underexplored problem. To solve these problems, we propose a framework of fast Imbalanced Kernel k-Means (IKKM), exploring both computational demands and statistical analysis. According to the theoretical analysis, the proposed fast IKKM can take less time to attain a similar accuracy of exact IKKM, when operating with a sketching dimension of approximately Ω(√(n)) with n denoting the sample count. In particular, we establish the first optimal excess clustering risk bound for the fast IKKM under mild conditions. Comprehensive experiments validate the theoretical analysis of the fast IKKM in addressing the computational challenges of large-scale imbalanced clustering.

Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood. In this work, we reveal that hallucinations are strongly associated with a human-like attention distraction phenomenon, where humans under divided focus experience degraded visual clarity and produce inaccurate descriptions, while in models the same mechanism manifests as spatial inconsistency in multi-head attention and temporal fading of attention to image tokens during decoding. We further provide theoretical insights that attention dispersion increases model complexity and degrades classification generalization. Motivated by these findings, we propose an Attention-Focused Approach for Improved Image Perception (AFIP), which corrects attention distraction via cross-head attention enrichment and reinforces visual grounding through dynamic historical attention enhancement. Extensive experiments on multiple benchmarks and models validate the effectiveness of AFIP without additional training.

Multi-instance multi-label learning (MIML) has emerged as a powerful paradigm for modeling complex objects with multiple semantics. In practice, the increasing demand for interpretability and precise attribution in tasks, such as malignant tissue region recognition in medical image diagnosis and posture transition monitoring in eldercare, necessitates the simultaneous implementation of coarse-grained and fine-grained annotations. However, the naturally skewed label distribution and label dependence introduce the issue of skewed annotation in dual-granularity labeling, which is crucial but rarely explored. To address this problem, we propose a non-convex dual-granularity labeling method for multi-instance multi-label data with skewed annotation named IMIMLdl. Specifically, we introduce the instance weight vector with a non-convex sparsity-inducing regularization to recognize key instances and obtain more discriminative object representation. To handle skewed label distribution and label correlation simultaneously, IMIMLdl cooperates with the re-weighted loss and low-rank regularization, enhancing model capability to recognize minority label classes and capture correlation. Meanwhile, we design an iterative accelerated block coordinate descent algorithm with proven convergence to solve our formulated non-convex and non-smooth problem. Finally, extensive experimental results validate the effectiveness of IMIMLdl against state-of-the-art methods, and a human activity recognition application demonstrates its potential to tackle real-world problems.

The label shift hypothesis posits that the source and target domains exhibit differing label priors while maintaining identical conditional probabilities. This core assumption enables models to adapt to domain distribution discrepancies through loss re-weighting or classifier output calibration, making it a prominent focus in open-environment machine learning. However, the prevalent existence of label noise severely degrades the classification efficacy of source classifiers, fundamentally undermining traditional label shift adaptation approaches. To address the dual challenges of label shift and label noise, we propose the Denoising Expectation-Maximization (DNEM) framework, which enhances model robustness against distribution shifts and label noise via iterative estimation of noise transition matrices and optimization of importance weights. Leveraging the memorization effect in deep learning, DNEM dynamically adjusts the learning rate for noise matrix estimation, maintaining conservative updates during initial training phases to preserve knowledge from relatively clean labels. Extensive experiments on multiple benchmark datasets demonstrate that DNEM consistently outperforms state-of-the-art methods under high noise rates and diverse shift conditions.

Partial Label Learning (PLL) aims to train multi-class classifiers from examples where each instance is associated with a set of candidate labels, among which the ground-truth label is assumed to be included. While most existing studies assume that partial labels are both instance-independent and reliable, such assumptions often break down in real-world scenarios, where candidate sets may depend on instance-specific features and even exclude the ground-truth label. In this work, we investigate a more realistic setting termed Unreliable Instance-Dependent Partial Label Learning (UIDPLL). To address the challenges in UIDPLL, we propose a novel framework named Neighborhood-guided Label Augmentation and Pruning (NLAP). NLAP exploits the structural consistency among neighboring instances to progressively refine candidate label sets and integrates classifier feedback to disambiguate labels during training. This progressive mechanism improves classification performance by tackling ambiguity caused by noise and instance dependency in partial labels. Furthermore, we provide theoretical guarantees for the proposed NLAP framework, demonstrating that label ambiguity can be effectively reduced through appropriate refinement and pruning procedures. Extensive experiments on both benchmark and real-world datasets demonstrate the robustness and effectiveness of the proposed method.

Transfer subspace learning plays a critical role in unsupervised domain adaptation by establishing a shared embedding space where source domain data can be linearly reconstructed to match target domain distribution. While existing methods exploit the low-rank structures of reconstruction matrix, they frequently overlook the alignment of cross-domain joint probability distributions in the learned low-dimensional subspace. To address these challenges, we propose a novel non-convex transfer learning method named DATSL, which employs embedded distribution alignment. Our DATSL incorporates a non-convex regularizer to approximate low-rank constraints, capturing the complex characteristics of the rank function by minimizing top $k$ smallest singular values of reconstruction matrix. To align the joint distributions across domains, a category-aware joint distribution alignment mechanism extracts more discriminative representations and enhances subspace discriminability through label-informed covariance matching. Besides, DATSL is extended to a graph-based variant GDATSL, which incorporates manifold-preserving constraints via Laplacian regularization to maintain intrinsic data topology during knowledge transfer. Furthermore, we develop an efficient iterative optimization algorithm to solve our formulated nonconvex minimization problems with proved convergence. Extensive experimental results on several public datasets demonstrate the effectiveness of our proposed methods in comparison to other state-of-the-art approaches.

Bipartite graph-based multi-view clustering conducts clustering of samples in accordance with the relationships between samples and anchors, and has demonstrated significant advancements in recent years. Predefined bipartite graphs with fixed anchors may not reflect the underlying clustering structure accurately, leading to the degradation of clustering performance. To address this problem, we propose a Structure Sparsity-Induced Bipartite Graph (SSBG) learning method to dynamically construct view-specific bipartite graphs with automatically learned anchors. Concretely, representative anchors of each view are learned by integrating key samples selected by introducing a selection matrix with structure sparsity. Meanwhile, the feature matrix of each view is reconstructed by the learned anchors and the corresponding bipartite graph in a self-representation manner. Due to the representativeness of the anchors and the advantages of the self-representation model in representing complex relationships, the consistent bipartite graph fused from multiple views possesses enhanced ability to represent the underlying clustering structure. A converged iterating algorithm is developed to optimize for the objective function, and the final clustering partition can be directly obtained according to the connected components of the fused consistent bipartite graph. Extensive experimental results demonstrate the advantages of SSBG in clustering performance across various benchmark datasets.

Clear sample correspondence across views is a key presupposition of traditional multi-view clustering. However, in practical applications, uncertainties during the data collection process may lead to the violation of this presupposition, producing unaligned multi-view data. In this paper, to overcome the obstacle of multi-view fusion caused by unaligned samples and achieve efficient unaligned multi-view clustering, a novel Diversified Anchor Graph Fusion (DAGF) method is proposed. Specifically, view-specific bipartite graphs with diversified anchors are constructed to adapt to the characteristics of unaligned multi-view data. Then, with the devised sample alignment and anchor integration strategy, these bipartite graphs are fused to learn a joint bipartite graph with explicit cluster membership structure. The proposed DAGF method not only overcomes the adverse effects of unaligned samples on cross-view information fusion, but also preserves complementary view-specific clustering structure information, enabling efficient and effective clustering. Systematic experimental results on real-world datasets demonstrate the advantages of the DAGF method in both clustering performance and computational complexity. Code available: https://github.com/revolution6575/DAGF.git.

In many learning systems, such as activity recognition systems, as new data collection methods continue to emerge in various dynamic environmental applications, the attributes of instances accumulate incrementally, with data being stored in gradually expanding feature spaces. How to design theoretically guaranteed algorithms to effectively cluster this special type of data stream, commonly referred to as activity recognition, remains unexplored. Compared to traditional scenarios, we will face at least two fundamental questions in this feature incremental scenario. (i) How to design preliminary and effective algorithms to address the feature incremental clustering problem? (ii) How to analyze the generalization bounds for the proposed algorithms and under what conditions do these algorithms provide a strong generalization guarantee? To address these problems, by tailoring the most common clustering algorithm, i.e., k-means, as an example, we propose four types of Feature Incremental Clustering (FIC) algorithms corresponding to different situations of data access: Feature Tailoring (FT), Data Reconstruction (DR), Data Adaptation (DA), and Model Reuse (MR), abbreviated as FIC-FT, FIC-DR, FIC-DA, and FIC-MR. Subsequently, we offer a detailed analysis of the generalization error bounds for these four algorithms and highlight the critical factors influencing these bounds, such as the amounts of training data, the complexity of the hypothesis space, the quality of pre-trained models, and the discrepancy of the reconstruction feature distribution. The numerical experiments show the effectiveness of the proposed algorithms, particularly in their application to activity recognition clustering tasks.

In multi-label learning applications, increasing annotation costs and data collection difficulties often result in incomplete labels, which significantly degrade classification accuracy. Traditional solutions face three key limitations in handling incomplete labels. First, convex optimization approaches struggle to capture label correlations, resulting in lower recovery accuracy than nonconvex models. Second, while nonconvex models show promise, they present significant theoretical and computational challenges, particularly in generalization analysis. In addition, missing labels exacerbate class imbalance issues. Our study revealed that multi-label classification performance is fundamentally bounded by and sensitive to decision thresholds. Fixed or manual thresholds typically result in suboptimal performance. To address these challenges, this study proposes a nonconvex and adaptive multi-label learning for highly incomplete labels (NAMLL). Specifically, to preserve the compact information of label correlations with missing labels, our NAMLL employed a Schatten-p norm regularization and maximum likelihood estimation into our model. Meanwhile, according to the threshold analysis, we adaptively adjust the optimal threshold to enhance its robustness to class imbalance. Moreover, we develop an efficient iterative algorithm to solve the proposed nonconvex objective and present the theoretical analyzes of its generalization bound and convergence of NAMLL. Extensive experimental results show NAMLL outperformed state-of-the-art methods across multiple metrics.

Second-order Online Kernel Learning (OKL) has attracted considerable research interest due to its promising predictive performance in streaming environments. However, existing second-order OKL approaches suffer from at least quadratic time complexity with respect to the pre-set budget, rendering them unsuitable for large-scale datasets. Moreover, the singular value decomposition required to obtain explicit feature mapping is computationally expensive due to the complete decomposition process. To address these issues, we propose FORKS, a fast incremental matrix sketching and decomposition approach tailored for second-order OKL. FORKS constructs an incremental maintenance paradigm for second-order kernelized gradient descent, which includes incremental matrix sketching for kernel approximation and incremental matrix decomposition for explicit feature mapping construction. Theoretical analysis demonstrates that FORKS achieves a logarithmic regret guarantee on par with other second-order approaches while maintaining a linear time complexity w.r.t. the budget, significantly enhancing efficiency over existing methods. We validate the performance of our method through extensive experiments conducted on real-world datasets, demonstrating its superior scalability and robustness against adversarial attacks.

Time-dependent partial differential equations (PDEs) are important mathematical tools for studying nonlinear dynamics. The recently emerged machine learning methodologies, such as Physics-Informed Neural Networks (PINN), offer an appealing alternative to traditional PDE solvers. These approaches are distinguished by their advanced flexibility and the effortless integration of observed data. However, when solving complex PDEs, especially time-dependent ones, PINN encounters training difficulties, manifesting in convergence to nonphysical solutions. This may be due to the insufficient capability of PINN to maintain the correct time evolution, leading to the difficulty in preserving the accuracy of solutions at later times, even if the initial values are correct. To lessen training difficulties, we aim to enhance PINN’s utilization of temporal information, thereby strengthening its capability to converge in accordance with the underlying physical dynamics of PDEs. Inspired by the temporal marching inherent of traditional numerical methods, we develop the dual-level time-marching neural network. For the outer time-marching level, we employ a neural network to progressively learn the solution in divided temporal sub-domains. For the inner one, as the model learns the solution within the current sub-domain, an implicit pseudo time-stepping loss is used to guide the model to converge along the trajectory of temporal evolution. The outer and inner levels of time-marching mechanisms are both essential in enhancing the model’s ability to handle time evolution. By investigating the spatio-temporal dynamics of a series of time-dependent PDEs, our proposed method is demonstrated to offer accurate predictions and significantly enhance both accuracy and efficiency when compared to PINN and two related improved methods.

Applying current machine learning algorithms in complex and open environments remains challenging, especially when different changing elements are coupled and the training data is scarce. For example, in the activity recognition task, the motion sensors may change position or fall off due to the intensity of the activity, leading to changes in feature space and finally resulting in label noise. Learning from such a problem where the dynamic features are coupled with noisy labels is crucial but rarely studied, particularly when the noisy samples in new feature space are limited. In this paper, we tackle the above problem by proposing a novel two-stage algorithm, called Adaptive Learning for Dynamic features and Noisy labels (ALDN). Specifically, optimal transport is first modified to map the previously learned heterogeneous model to the prior model of the current stage. Then, to fully reuse the mapped prior model, we add a simple yet efficient regularizer as the consistency constraint to assist both the estimation of the noise transition matrix and the model training in the current stage. Finally, two implementations with direct (ALDN-D) and indirect (ALDN-ID) constraints are illustrated for better investigation. More importantly, we provide theoretical guarantees for risk minimization of ALDN-D and ALDN-ID. Extensive experiments validate the effectiveness of the proposed algorithms.

Label shift, which investigates the adaptation of label distributions between the fixed source and target domains, has attracted significant research interests and broad applications in offline settings. In real-world scenarios, however, data often arrives as a continuous stream. Addressing label shift in online learning settings is paramount. Existing strategies, which tailor traditional offline label shift techniques to online settings, have degraded performance due to the inconsistent estimation of label distributions and violation of convex assumption for theoretical guarantee. In this paper, we propose a novel method to ensure consistent adaptation to online label shift. We construct a new convex risk estimator that is pivotal for both online optimization and theoretical analysis. Furthermore, we enhance an optimistic online algorithm as the base learner and refine the classifier using an ensemble method. Theoretically, we derive a universal dynamic regret which achieves minimax optimal. Extensive experiments on both real-world datasets and human motion task demonstrate the superiority of our method comparing existing methods.