Positive-Unlabeled (PU) learning focuses on scenarios where only positive and unlabeled examples are available and has become a crucial area in machine learning. However, its application to data enriched with structured or symbolic knowledge remains unstudied. Existing PU learning methods often rely on the assumption of irreducibility between negative and positive distributions, but this assumption has not been studied in settings where the data is governed by logical rules or relational constraints. In this work, we formalize PU learning for knowledge-driven data and introduce a novel approach that leverages a neuro-symbolic framework to address the PU learning problem in such data settings. We further propose NeuralAdjust, a method that uses neural networks to dynamically reweight logical rules. This helps mitigate violations of the irreducibility condition induced by relational conflicts. Our experimental results demonstrate robust performance across diverse datasets and knowledge structures, surpassing previous state-of-the-art graph-based methods for PU learning. https://github.com/haiyenvu96/NeuralAdjust-for-NeuPSL-PU .
Large language models (LLMs) can reproduce memorized sensitive data, including SSH keys, API tokens, and personally identifiable information (PII), when prompted with the right context. Removing these secrets is difficult: gradient-based unlearning often damages model utility, while preference-based methods can leave secrets recoverable, especially at larger scales. We show that a central reason is type heterogeneity: different secret types exhibit different gradient-saliency patterns and require different correction strengths. We introduce Per-Secret-Type Unlearning (PSTU), a training-free task-arithmetic method that replaces a single global scaling factor with per-type saliency and layer-group corrections. For 7B+ models, PSTU-Trim denoises the task vector before subtraction. Across our evaluated Pythia (1.4B to 6.9B) and Llama-3.1-8B configurations, PSTU removes all 175 measured memorized secrets with 1.3 ±1
Tree-based models decision trees, random forests, XGBoost, LightGBM, AdaBoost, dominate machine learning on tabular data, yet their formal interpretation remains intractable in general, unless the number of trees is fixed. We introduce k-pseudo-probable explanations, a new type of formal explanation for weighted tree-based ensembles: a subset of at most k features minimizing a weighted error function over the trees voting for the predicted class, where weights reflect each tree’s prediction confidence. This problem is NP-hard in general, but admits a supermodular structure enabling efficient approximation via two greedy algorithms, ga and gd, with theoretical guarantees based on the curvature of the objective. Our approach rests on the class-separation of implicants property, which guarantees semantic consistency and justifies restricting attention to trees voting for the predicted class. Experiments on benchmarks demonstrate the computational efficiency of our algorithms and the practical utility of the generated explanations.
Incremental Named Entity Recognition (INER) aims to learn new entity types from sequentially arriving data while preserving performance on previously learned ones. Existing methods often combine pseudo-labeling with knowledge distillation, but suffer from a fundamental issue: semantic boundary drift. As new entity types appear, the decision boundaries of old entity types are gradually distorted, being pushed toward the non-entity type or pulled into neighboring entity types due to unreliable pseudo-labels and static prototype representations. In this paper, we identify two distinct forms of such drift: non-entity attraction and cross-entity attraction, and propose a unified Boundary-Aware Calibration (BAC) framework to tackle them. First, we introduce choosy pseudo-labeling and confidence-weighted feature distillation to filter ambiguous tokens and suppress error propagation, mitigating drift toward the non-entity type. Second, we design hybrid-prototype guided class re-adaptation, which constructs confidence-aware hybrid prototypes by combining memory prototypes from the old model with adaptive prototypes from the current model, and enforces boundary-aware contrastive alignment. This keeps old-entity-type prototypes dynamically calibrated with the evolving feature space and decision boundaries, preventing them from being attracted by new entity types. Extensive experiments on Few-NERD, I2B2, and OntoNotes5 demonstrate that our method consistently outperforms state-of-the-art approaches. Code will be released.
Continual unlearning aims to learn a sequence of tasks over time without catastrophic forgetting while removing specified previously learned tasks on request without causing catastrophic unlearning, inheriting challenges from both continual learning and machine unlearning. Existing continual learning approaches mostly mitigate catastrophic forgetting but do not support controlled task removal. Meanwhile, neural networks face the network capacity problem and the stability-plasticity dilemma in continual learning, which make it increasingly difficult to maintain learning performance as tasks accumulate, especially in long task sequences. In this paper, we propose a general continual unlearning framework, Amnesiac Continual Learning (AmnesiacCL), which can be integrated into any continual learning approach. AmnesiacCL performs update deletion that can recycle network capacity to passively address this problem. On top of this, we propose Amnesiac Hard Attention to the Task (AmnesiacHAT), a continual unlearning approach that extends the architecture-based Adaptive Hard Attention to the Task (AdaHAT) continual learning approach. By reducing task interference, AmnesiacHAT leads to less catastrophic unlearning; we further mitigate it with two post-unlearning processes: backup compensation and replay repairing. Experiments on standard continual learning benchmarks demonstrate that AmnesiacHAT can effectively perform unlearning without degrading performance on remaining tasks, and that AmnesiacCL generally better balances the stability-plasticity trade-off in long task sequences through capacity recycling. Our results suggest that unlearning can serve not only to meet privacy and trustworthiness requirements but also as a passive mechanism benefiting continual learning by addressing the network capacity problem. Our code is available (pengxiang-wang.com/projects/continual-learning-arena).
Temporal graphs are a variety of structured data that have been particularly studied recently. In this paper, we focus on classification of temporal graphs. Previous works have shown that such task can be tackled by first transforming temporal graphs to static ones using ad-hoc algorithms, before the classification step. However, resulting static graphs might be of huge size. Transitive reduction can be used to reduce the size and redundancy of such graphs, but with a high complexity time. In this paper, we propose a fully dynamic algorithm that transforms temporal graphs into static ones while applying exact transitive reduction at the same time. This algorithm has a much reduced complexity than doing both steps separately, while producing exactly the same output graphs. We show in the experimental section that similar accuracy is achieved for a lower computational time.
Partial Multi-label Learning (PML) is a typical weakly supervised learning paradigm. In the complex label noise environment, PML models need semantic correlation features to build an adaptive perception for heterogeneous noise levels of labels. Nevertheless, most existing mainstream PML methods adopt a fixed-threshold strategy for label information propagation, and this strategy exacerbates the bottleneck of label ambiguity, making it unable to adapt to differentiated sample noise scenarios. To address the above situation, this paper proposes a novel label correlation-driven partial multi-label learning algorithm with dynamic noise threshold learning (PML-LE). The PML-LE realizes the embedding of label semantic features and correlation features through the label correlation modeling module to provide semantic support for the subsequent label disambiguation process. Meanwhile, PML-LE designs an adaptive noise thresholding mechanism based on the information entropy of candidate label sets, which dynamically adjusts the pruning strictness of label adjacency matrices according to each sample’s noise level. Experimental results on benchmark datasets demonstrate that the proposed PML-LE algorithm achieves state-of-the-art performance for PML.
Explanation-guided learning (EGL) improves the transparency of natural language processing (NLP) models by using token-level explanations as auxiliary supervision. However, explanations produced by modern NLP models are often highly sensitive to small, semantics-preserving perturbations: attribution patterns can shift toward spurious cues even when the underlying meaning remains unchanged. This instability weakens EGL because the explanatory signal itself becomes unreliable, and it is closely related to adversarial vulnerability in sentiment classification. In response, we propose HATS (Hybrid Adversarial Training guided by Saliency), a training framework that jointly promotes prediction robustness and explanation stability under hybrid adversarial perturbations. HATS generates adversarial counterparts from two complementary views: (i) continuous semantic-level perturbations in embedding space and (ii) discrete surface-level perturbations in token space via word/character transformations. In addition to the standard adversarial classification loss, HATS introduces two saliency-aware objectives: an explanation discrepancy loss that penalizes divergence between attribution distributions of clean and adversarial inputs, and a salient-evidence sufficiency loss that retains only salient tokens in the adversarial input while matching the resulting predictive distribution to that of the clean input. Experiments on three sentiment benchmarks show that HATS improves both adversarial accuracy and F1 by up to 16.6
Noisy labels are ubiquitous in real-world datasets, causing deep neural networks to overfit and suffer from poor generalization. Despite label corruption, prior works observe that during the “early learning” phase, learned representations of images from the same category still congregate together. Digging deeper into this phenomenon, we propose a novel framework to mitigate noisy supervision by creating synthetic samples on Riemannian manifolds. Unlike Euclidean methods that ignore intrinsic data geometry, our approach synthesizes features by aggregating original samples with their top-K neighbors along the geodesic paths, where the aggregation weights are determined by modeling the loss distribution. These synthetic samples serve as denoised proxies that effectively smooth the decision boundary and prevent the memorization of erroneous labels. Additionally, these synthesized representations estimate soft targets, progressively correcting noisy labels and yielding more separated, clearly bounded clusters. Experiments on different datasets demonstrate that our method outperforms state-of-the-art approaches, enhancing the robustness of learned representations.
Learning predictive models from functional magnetic resonance imaging (fMRI) connectivity data is important for analyzing cognitive processes and neurological disorders. Most existing approaches adopt a single-task learning (STL) formulation, where models predict either cognitive task states or clinical diagnoses independently. This setting often limits generalization because related tasks share latent neural representations that remain unused. Multi-task learning (MTL) addresses this limitation by learning shared representations across tasks, enabling complementary information transfer and improved robustness. In this work, we propose the Brain Connectivity Transformer (BCT), a graph-based transformer architecture for joint cognitive task and disorder classification from fMRI connectivity graphs. BCT operates on region-of-interest (ROI) connectivity graphs and employs multi-head self-attention to model global interactions among brain regions. A dynamic ROI attention module identifies task- and disorder-relevant regions, while a weighted orthonormal clustering readout aggregates attention-refined features into cluster-level representations. Within the MTL framework, BCT jointly optimizes task and disorder objectives. Under subject independent evaluation, BCT achieves 85.13± 0.3% task accuracy and 95.30± 1.1% disorder accuracy, outperforming the strongest baseline by 4.51% and 7.08% , respectively. These results demonstrate the effectiveness of transformer-based MTL on connectivity graphs for joint cognitive task and disorder prediction. The source code is available at https://github.com/DDasChakladar/Brain-Connectivity-Transformer .
Continual learning aims to enable models to learn a sequence of tasks without catastrophic forgetting, a phenomenon where new information overwrites previously acquired knowledge. Traditional solutions for this problem, including regularization, replay buffers, and dynamic architectures, struggle with trade-offs in scalability, privacy, and adaptability. Prompt-based learning, initially developed in NLP, offers parameter-efficient alternatives by prepending learnable vectors to input tokens. However, existing prompt methods in continual learning, such as L2P and DualPrompt, rely on deterministic selection mechanisms that lack uncertainty modeling, making them less effective in dynamic and ambiguous task scenarios. In this work, we propose a novel framework called Variational Inference based Probabilistic Prompt (VPrompt) that introduces a stochastic latent variable formulation over prompt selection using variational inference. Our method learns an approximate posterior distribution over prompt assignments conditioned on inputs, and regularizes this with a uniform prior to ensure diversity and mitigate overconfidence. This probabilistic mechanism enables uncertainty-aware adaptation, improves robustness under domain shift, and eliminates the need for task labels or rehearsal buffers. We evaluate our method across Split CIFAR100, Split ImageNet-R, and a diverse 5-dataset benchmark. VPrompt consistently outperforms state-of-the-art baselines, including CODA-Prompt, L2P, DualPrompt, regularized and rehearsal-based methods, in terms of average accuracy and reduced forgetting. These results confirm that modeling uncertainty at the prompt level offers a scalable, buffer-free, and more flexible solution for continual learning.
Learning from incomplete heterogeneous tabular data is challenging because missingness often exhibits structural dependencies across samples and features rather than occurring independently. Viewing tabular data as a partially observed bipartite graph, missing entries correspond to removed edges and structured missingness manifests as sparsification patterns in the sample–feature interaction graph. To address this problem, we propose IVGAE, a variational graph framework that jointly models latent data representations and missingness mechanisms. IVGAE performs variational inference over the bipartite graph to capture structural dependencies among heterogeneous features while modeling missingness as a graph-level generative process. A dual-decoder architecture simultaneously reconstructs feature values and infers missingness patterns under MCAR, MAR, and MNAR regimes. To effectively encode heterogeneous attributes, we introduce a Transformer-based feature embedding module that preserves categorical semantics without relying on high-dimensional one-hot representations. Experiments on 16 real-world datasets demonstrate that IVGAE consistently improves reconstruction accuracy and downstream prediction performance compared with strong imputation baselines across diverse missing regimes. The codes are available at: https://github.com/echoid/IVGAE .
Deep multi-label classifiers are often learned on partially labeled datasets, where only a small proportion of labels are annotated. While most existing works aim to attain high classification performance, calibration, a crucial property of classifiers in life-critical applications that measures whether the predicted probabilities can reflect the true likelihood of each category, has received very little attention. We found that the label imbalance-aware asymmetric (ASY) loss produces poorly calibrated classifiers, which may aggravate the generation of unwanted wrong pseudo-labels in pseudo-labeling approaches. In this paper, we propose an interactive learning framework that takes advantage of pseudo-labeling and the ASY loss to train accurate, calibrated classifiers, without threshold tuning and prior knowledge of the data. A dedicated neural network calibrates the classifier’s predicted probabilities for pseudo-label generation, mitigating the calibration problems of the ASY loss. The classifier and calibrator improve each other to frequently update pseudo-labels. Additionally, we propose a post-training method that learns an element-wise linear regression model re-parameterizable into the classifier, enabling the classifier to directly produce calibrated probabilities without extra inference costs. Extensive experiments are conducted on nine partially labeled datasets derived from MS-COCO, VG-200, and NUS-WIDE. We achieve 2.9x-8.7x more mAP improvements than the best of the nine compared pseudo-labeling methods, with significantly lower calibration errors.
In this study, we identify critical problems in graph similarity computation: context deficiency and spurious alignments. To solve these problems, our approach regards motifs as fundamental alignment units for capturing essential local structures while maintaining semantic context, and proposes a two-stage learning paradigm, involving Self-Supervised Motif Extraction and Motif Interaction Graph Similarity Learning. Moreover, we design multiple structural regularizations to ensure that these motifs retain their semantically meaningful properties (confidence, balance, diversity, connectivity). Theoretically, we prove that motif-level matching achieves a lower error upper bound, establishing that motif-level alignment is provably more robust than node-level alternatives. Extensive experiments on three benchmark datasets—LINUX, AIDS700nef, and IMDBMulti—demonstrate that MEGSA consistently outperforms state-of-the-art baselines across multiple evaluation metrics, achieving improvements of 9.73
Many real-world networks can be modeled as graphs, where identifying dense subgraphs is a fundamental task in graph mining. In this paper, we extend the densest subgraph problem, by seeking multiple disjoint dense subgraphs rather than a single solution. More formally, given a graph and an input parameter k, our goal is to find k disjoint subgraphs maximizing the minimum induced density of the k subgraphs. We prove that this problem is NP-hard even for k=2 . To tackle this, we present an exact algorithm based on mixed integer programming and propose 3 quadratic-time heuristics using dynamic programming. To ensure scalability for large-scale datasets, we introduce linear-time variants of these heuristics by limiting the sizes of the subgraphs. Our extensive experimental study compares these approaches against two baseline greedy heuristics. The results demonstrate that our algorithms accurately recover ground truth in synthetic datasets and produce good solutions in real-world datasets when compared to the baselines.
Multi-modal Knowledge Graph Completion aims to predict missing semantic links between entities by integrating structured knowledge with multi-modal information. However, existing methods typically treat different modalities as independent feature sources, employing simple concatenation or late fusion while overlooking the deep semantic correlations between visual and textual modalities. Furthermore, latent noise within multi-modal data often prevents effective alignment with the structural topology of the Knowledge Graph, thereby undermining reasoning performance. To address these limitations, we propose a Structure-Guided Fusion Semantic Denoising(SgFSD) model. Specifically, to achieve cross-modal semantic interaction and mitigate information loss during fusion, we design a Multi-modal Semantic Fusion module that enables deep token-level interactions between visual and textual modalities. And an adaptive gated weighting mechanism is further introduced to dynamically regulate the contribution of original modality features, producing high-quality fused representations. To suppress residual noise introduced during the fusion process and improve cross-modal consistency, we further incorporate a Structure-Guided Contrastive Learning module, which leverages structural embeddings from the knowledge graph as reliable anchors to constrain and align the fused multi-modal representations. This structure-guided denoising strategy effectively reduces noise interference and enhances representation consistency. Based on the entity representations, a unified scoring and prediction module is employed to perform knowledge graph completion. Extensive experiments on three public benchmark datasets demonstrate that SgFSD significantly outperforms baselines, validating the effectiveness of our multi-modal semantic fusion and structure-guided denoising strategies.
Geometry-aware models capturing the inherent structure of data often allow for improved performance, robustness, and generalisation when learning representation spaces. Established approaches, however, require major changes in the model architecture and render the integration of non-standard geometries rather involving. In this paper, we propose a novel regulariser that maps data on a Riemannian manifold of choice where geodesic balls are placed around instances and their overlaps penalized. Minimizing the regulariser requires minimizing these overlaps and, hence, our approach promotes separability and leads to meaningfully structured latent spaces while leaving the model architecture untouched. Empirically, we observe improved performance on a variety of settings.
Multimodal Relation Extraction (MRE) identifies semantic relations between entity pairs by integrating textual and visual information. Existing methods typically employ a unified classification or single-retriever paradigm, applying one model across all relation types. However, relation evidence is inherently heterogeneous: some relations are text-dominant, others rely on visual cues, and many require cross-modal reasoning. Such diversity introduces noise and limits the effectiveness of monolithic architectures. Although retrieval-based approaches alleviate some issues, they still depend on a single retriever, failing to capture modality-specific evidence patterns. To address these limitations, we propose Bayesian Mixture-of-Retrievers with Evidence-enhanced Cycle-consistency (BMoR-EC), which incorporates three specialized retrievers—text-modal, image-modal, and hybrid—to capture complementary modality-specific evidence. An evidence-enhanced cycle-consistency constraint models the logical dependencies among entities, evidence, and relations. Moreover, a Bayesian fusion mechanism with evidence-dependent priors dynamically weights retriever outputs according to estimated reliability. The framework is jointly optimized with multi-dimensional retrieval losses and requires no explicit evidence at inference. Experiments on MNRE and MORE demonstrate improved robustness and reduced retrieval noise across diverse relation types.
Multi-agent trading systems built on large language models (LLMs) are beginning to appear in quantitative finance, yet their robustness to adversarial inputs is largely unknown. We study the vulnerability of LLM trading stacks to black-box, input-only attacks that enter solely via admissible social-media feeds. We introduce the Generic Multi-Agent Trading System (GMATS), a framework that captures modern multi-agent trading architectures and instantiate a class of black-box poisoning attackers that treat an LLM as a post generator and inject budget-constrained, plausibly benign social-media content into the analyst’s evidence stream. We define contagion metrics that trace how adversarial content propagates through the stack, including belief-shift scores at analyst and coordinator layers and attack–clean deltas on standard backtest metrics. Experiments on a safe offline benchmark with historical market and social data show that even simple input-only attackers can materially degrade risk–return profiles, sharply reducing Sharpe ratios. At the same time, we find that suitably designed multi-agent topologies and coordinator prompts can dampen adversarial shocks and improve average robustness under identical poisoning budgets. Our code and dataset are available in this repository .
The rapid deployment of artificial intelligence (AI) across sectors exposes organizations to overlapping and mutually reinforcing risks spanning cybersecurity, privacy, and AI‑specific dimensions. Threats such as inference attacks, adversarial manipulation, and privacy leakage cut across all three domains yet are typically addressed in isolation, resulting in fragmented assessments, duplicated effort, and regulatory blind spots. The European regulatory landscape encompassing the GDPR, the EU AI Act, NIS 2, and the forthcoming Cyber Resilience Act, further compounds this fragmentation by imposing distinct, domain-specific risk management obligations that organisations must reconcile in practice. This paper addresses these challenges by proposing an integrated conceptual model that systematically unifies key notions from the AI, privacy, and cybersecurity risk domains. Grounded in the Information System Security Risk Management (ISSRM) domain model and extended through structured semantic alignment with relevant international standards and regulatory guidelines, the model enables coherent, non-redundant risk assessments across the three domains. It has been implemented in a model-based risk management platform, and an evaluation through a case study, conducted in collaboration with RMT Labs, a provider of an AI-supported recruitment system, has been initiated.
