Federated learning (FL) over low Earth orbit (LEO) satellite constellations faces a fundamental safety challenge. Intermittent satellite visibility causes standard client scheduling methods to systematically exclude certain geographic regions, producing ground stations with zero recall that render entire populations invisible to safety critical event detection. This work demonstrates that existing schedulers, including random selection, loss based prioritization, and operational heuristics such as maximum visibility and geographic diversity, routinely produce this catastrophic failure mode under realistic orbital dynamics, yet the failure remains hidden when only aggregate metrics are reported. Standard evaluation, relying solely on aggregate metrics such as average test accuracy and convergence speed, conceals catastrophic localized failures that disaggregated per node safety metrics reveal. To substantially reduce this failure, a graph spectral scheduling framework called Graph Centric Periodic Scheduling (GCPS) is introduced. GCPS preserves the algebraic connectivity of the training subgraph by jointly optimizing betweenness centrality, participation diversity, eigenvector contribution to the Fiedler vector, and node degree, with weights dynamically adapted through a validation only multi armed bandit exploring seven Pareto optimal weight configurations. The central finding establishes that GCPS substantially reduces zero recall nodes to 0 out of 5 trials (0 p < 0.01 versus geographic diversity, graph weighted sampling, q-FFL, AFL, and Oort), maintains competitive standard accuracy, and demonstrates robustness to topology mismatch, differential privacy noise, and label flipping attacks. Beyond the specific satellite domain, this work argues that minimum per node performance guarantees must become a primary evaluation criterion for safety critical FL, as aggregate metrics conceal catastrophic localized failures that graph spectral scheduling is uniquely positioned to prevent. It is further demonstrated that commonly used fairness indices, with Jain’s index exceeding 0.98 for all methods, measure participation equity rather than performance equity, and therefore fail to detect zero recall failures, a critical distinction for safety critical deployment.
Equation discovery has traditionally focused on regression, where the goal is to recover analytical expressions that model numerical targets. In this paper, we extend this paradigm to binary classification and introduce Equation Discovery for Classification (EDC), a framework that discovers concise symbolic expressions that explicitly define decision boundaries. EDC searches over a configurable grammar of analytical expressions using beam search and optimises equation parameters via dedicated numerical procedures, yielding a single interpretable equation whose sign determines class membership. We design a redundancy-aware grammar that balances expressivity and tractability, enabling the discovery of non-linear decision boundaries while maintaining interpretability. Through experiments on artificial datasets with known generating mechanisms, we show that EDC reliably reconstructs complex target boundaries, including XOR-like and interaction-driven structures, and adapts effectively under increasing levels of noise. Notably, in noisy settings EDC can outperform the original generating equation by approximating the implicit, noise-shifted decision boundary. On UCI benchmark datasets, EDC consistently outperforms existing symbolic classification approaches and other interpretable baselines, while achieving performance competitive with state-of-the-art black-box models. Although computationally more demanding than standard classifiers, we demonstrate that substantial speed-ups can be achieved with reduced search depth and simplified grammars at minimal loss of predictive performance. Overall, EDC provides a principled bridge between symbolic regression and classification, offering a transparent yet expressive alternative to black-box models for applications where interpretability of the decision boundary is essential.
The increasing use of machine learning algorithms in social applications has raised concerns about fairness and transparency, leading to the development of counterfactual explanations. These explanations support individuals to understand and potentially alter unfavorable decisions in areas such as loan applications, job selections, and more, by providing actionable changes to input features that would lead to a desired outcome. Existing methods often struggle to balance feasibility, plausibility, and computational efficiency. To address this, we introduce P ^2 CE, an algorithm for generating plausible Pareto-optimal counterfactual explanations, offering users a diverse set of optimal trade-offs between different notions of feasibility. P ^2 CE employs an auxiliary isolation forest outlier detector to ensure that explanations are in accordance with the data distribution and leverages SHAP values to obtain optimal results with short computing times, regardless of the underlying model. Our algorithm was empirically evaluated on three datasets, demonstrating superior performance in terms of both solution quality and computational efficiency compared to related techniques.
Large language models (LLMs) have achieved remarkable performance across a wide range of downstream tasks, with parameter-efficient fine-tuning methods such as LoRA significantly reducing computational overhead. However, LLMs remain vulnerable to catastrophic forgetting when learning sequential tasks, and the rank of LoRA is typically fixed at the same value across all model layers and tasks, leaving the role of rank underexplored in continual learning. In this paper, we propose a novel parameter-efficient approach for continual learning in LLMs, which adaptively adjusts layerwise LoRA ranks to help balance the stability–plasticity trade-off. Our proposed method leverages a subspace similarity metric to evaluate the orthogonality between task-specific low-rank subspaces and thus adaptively increases the layerwise rank for each new task, thereby minimizing interference with previously learned tasks while enhancing generalization. Experimental results on standard natural language processing continual learning benchmarks and challenging mathematical benchmarks show that our method achieves comparable or superior performance in mitigating forgetting, improving task performance, and maintaining strong generalization to unseen data in a memory-efficient manner. Code is provided in the link .
In this paper, we explore how multi-label classification (MLC) tasks can be cast into order structure learning. Our motivation for doing so is to exploit the very rich structure of the orders to improve and robustify MLC learning. We describe formally how MLC can be transformed into an order structure learning and prediction task, and then proceed to study the problem of predicting Bayes-optimal order structures. We then perform some experiments in settings where the use of order structures can be very beneficial: robust MLC in the presence of noisy and imbalanced labels, and making MLC predictions with partial abstention.
Accurate prediction for multimodal data—including tabular, textual, and visual inputs or outputs—is essential for advancing analytics across diverse application domains. Existing methods often struggle to integrate heterogeneous data types while maintaining strong predictive performance. We introduce Generative Distribution Prediction (GDP), a model-agnostic framework that leverages high-fidelity multimodal synthetic data generated from the conditional distribution of interest, such as via conditional diffusion models, to enhance prediction across both structured and unstructured modalities. GDP is compatible with any expressive generative model and naturally supports transfer learning for domain adaptation. We provide a rigorous theoretical foundation for GDP, establishing statistical guarantees on its predictive accuracy when diffusion models serve as the generative backbone. By estimating the underlying data-generating distribution and enabling loss-adapted risk minimization, GDP delivers accurate point predictions in broad multimodal settings. We empirically validate GDP on a range of supervised learning tasks, including adaptive quantile regression, modal regression, tabular prediction, image captioning, and question answering, demonstrating its versatility and effectiveness across domains.
Self-supervised learning (SSL) addresses label scarcity by leveraging large amounts of unlabeled data to learn transferable representations. However, pretraining on very large unlabeled datasets can be computationally expensive and may include noisy or unrepresentative samples that degrade learning. In this work, we explore whether selecting a subset of the available unlabeled examples for the pretext task can reduce computational cost while maintaining strong performance of SSL methods for tabular data. In particular, we investigate whether uncertainty-, diversity-, and transport-based criteria can guide this selection and improve representation quality. To this end, we conduct large-scale experiments on 25 tabular benchmark datasets using four SSL models, across varying amounts of labeled data under both biased and unbiased label selection, and multiple strategies for sampling unlabeled data. To better understand when and how unlabeled data subsampling is effective, we relate dataset characteristics to experimentally observed performance gains through a meta-analysis. Our results show that reducing the unlabeled data pool yields substantial computational savings while maintaining, and often even improving, downstream performance.
Machine learning models deployed in high-stakes domains such as healthcare must remain both accurate and robust to subtle, semantically meaningful temporal changes between classes. However, existing sensitivity-analysis methods for time series classifiers typically report aggregate robustness without validating whether their underlying metric reflects the position of a classifier’s decision boundary, and it is unclear whether this holds equally across different choices of the morphing operator such methods rely on to generate intermediate instances. We propose tsMIST (Time Series Model Sensitivity Test), a framework that morphs borderline pairs of real instances from different classes and summarizes, via two measures, tsMIST_Avg and tsMIST_Std , the point along the morphing path at which a classifier’s prediction switches class. We prove formally, and confirm on data with a known decision boundary, the conditions under which tsMIST_Avg estimates boundary position, showing that its commonly-assumed reference value of 0.5 reflects source-target symmetry rather than a universal target. We further compare linear interpolation against a path-interpolation (PathI) operator that slides matched features in time via their dynamic-time-warping correspondence rather than blending amplitudes. The results show that linear morphing is structurally unable to recover boundary position when the discriminative signal between classes is temporal rather than amplitude-based, whereas PathI recovers it exactly. On ten real-world medical time series datasets and four classifiers, InceptionTime consistently exhibits the most robust decision boundaries and Catch22 the most fragile; PathI and linear morphing differ systematically in estimated boundary position without a practically meaningful difference in consistency, and PathI more closely preserves the temporal structure of the original signal. We conclude that the choice of morphing operator materially affects sensitivity-analysis outcomes and should be justified and reported rather than assumed, supporting structured, boundary-aware sensitivity analysis as a component of Responsible AI evaluation.
Detecting anomalous inputs is a critical prerequisite for the safe deployment of reinforcement learning (RL) agents in real-world environments. Although anomaly detection (AD) has been extensively studied in other domains, its application to reinforcement learning remains challenging due to high-dimensional sensory observations and complex temporal dependencies. Existing approaches in this setting are limited and often rely on access to internal representations of trained agents, creating an undesirable coupling between policy and safety mechanisms. In this work, we propose ITRM, a novel approach to anomaly detection in visual reinforcement learning that is fully agent-agnostic and does not require access to policy internals. Our method is based on the observation that deviations from nominal environment dynamics can be identified through discrepancies in the latent representations of a learned world model. Specifically, we leverage predictive components of a recurrent state-space model to generate deterministic latent embeddings that serve as normative references for anomaly detection. Anomalies are detected by comparing predicted latent features against a reference set of nominal embeddings using a similarity-based criterion. Operating in the world model’s latent space allows the detector to capture semantically meaningful deviations. Extensive empirical evaluations and ablation studies demonstrate that the proposed approach achieves strong detection performance. On the Anomaly-Gym benchmark, our method outperforms existing baselines, achieving an average AUROC of 0.853 and an FPR95 of 0.279.
The rapid development of convolutional neural networks (CNNs) has led to an increase in their scale and complexity, thereby driving the rapid development of model pruning techniques. However, most existing methods suffer from the issue of being empirical and insufficiently explored when determining the pruning rate for each layer. To address the above issues, this paper proposes a dynamic pruning method that uses reinforcement learning guided by a dynamic reward function to automatically explore the pruning rate of each layer, and uses the designed similarity-based connected graph (SCG) mechanism to perform channel removal operations during training. First, a reward function with the accuracy and parameter count of current compressed CNNs was designed to guide pruning, and a dynamic balancing factor was introduced to continuously adjust the agent’s attention to both. Second, a series of connected subgraphs are created according to the SCG, and then representative channels are selected in different subgraphs according to importance criteria. Extensive experiments across multiple datasets demonstrate the effectiveness of this research. For instance, for ResNet56 on CIFAR-10, this work achieves a 61.63
One of the central problems studied in the theory of machine learning is the question of whether, for a given class of hypotheses, it is possible to efficiently find a consistent hypothesis, i.e., one with zero training error. While problems involving convex hypotheses have been extensively studied, the question of whether efficient learning is possible for non-convex hypotheses composed of possibly several disconnected regions is still not well understood. Although it has been shown quite a while ago that efficient learning of weakly convex hypotheses, a parameterized relaxation of convex hypotheses, is possible for the special case of Boolean functions, the question of whether this idea can be developed into a generic paradigm has not yet been studied. In this paper, we provide a positive answer and show that the consistent hypothesis finding problem can indeed be solved in polynomial time for a broad class of weakly convex hypotheses over metric spaces. To this end, we propose a general domain-independent algorithm for finding consistent weakly convex hypotheses and prove sufficient conditions for its efficiency that characterize the corresponding hypothesis classes. To illustrate our general algorithm and its properties, we discuss several non-trivial learning examples to demonstrate how it can be used to efficiently solve the corresponding consistent hypothesis finding problem. Without the weak convexity constraint, these problems are known to be computationally intractable. We then show that the general idea of our algorithm even extends to the extensional case, enabling applications, such as vertex classification in graphs. We prove that using our extended algorithm, the problem can be solved in polynomial time provided the distances in the domain can be computed efficiently.
Modeling the temporal evolution of biological systems is fundamental for understanding cellular dynamics and anticipating future functional states. While temporal graph neural networks (TGNNs) have achieved remarkable success in social and financial domains, their evaluation on dynamic biological systems remains largely unexplored. In this work, we provide the first systematic benchmark of discrete-time temporal graph neural networks on dynamic protein-protein interaction (PPI) networks, considering both future link prediction and future gene expression forecasting as complementary structure- and node-level tasks. To capture the recurring and synchronized nature of biological dynamics, we introduce State-Memory Temporal Graph Neural Networks (SM-TGNN), a novel architecture that augments message passing with a compact state-memory mechanism designed to model recurrent structural regimes without relying on sequential recurrent units. Across multiple yeast PPI datasets, SM-TGNN achieves consistently competitive performance in predicting future protein interactions and gene expression profiles, matching or exceeding existing neural approaches across most evaluation settings. At the same time, the strong results obtained by memory-based baselines indicate that temporal link prediction in dynamic biological networks remains a particularly challenging task, requiring models capable of capturing recurrent interaction regimes and long-term temporal dependencies. Notably, a model pre-trained on one PPI network achieves competitive performance when transferred to a distinct yeast cell-cycle dataset, suggesting that the learned state representations capture recurring temporal structures that can partially generalize across related biological settings. Furthermore, SM-TGNN offers competitive inference-time and memory efficiency compared to standard TGNN architectures. Our results demonstrate that state-based temporal modeling provides an effective and scalable inductive bias for learning from dynamic biological networks, opening new directions for temporal graph learning as an AI-driven simulation of cellular processes.
We study the problem of exact equation discovery, i.e., identifying symbolic equations that perfectly describe relationships in noise-free data. While most existing approaches focus on approximate recovery from noisy measurements, we consider settings in which exact correctness is required. This setting is closely related to methods that infer symbolic relations, such as recurrence equations or generating functions, from finite data. We show that exact equation discovery can be formulated as the computation of the vanishing ideal of the observed data and leverage Gröbner bases as an effective algorithmic tool. Building on this connection, we introduce MoadeeB, a new algorithm for discovering exact equations over integers and rational numbers. We evaluate MoadeeB in a large-scale empirical study on more than 30,000 integer sequences from the Online Encyclopedia of Integer Sequences (OEIS), focusing on the reconstruction of known recurrences and the discovery of previously undocumented ones. We compare against state-of-the-art symbolic regression and program synthesis approaches, as well as approaches from experimental mathematics and computer algebra that infer symbolic relations directly from finite sequence prefixes. The results show that MoadeeB achieves competitive or superior performance across these method classes, while additionally enabling the discovery of exact equations beyond the scope of existing approaches.
Faithfulness hallucinations, where large language models generate outputs unsupported by retrieved evidence, remain a central challenge for trustworthy AI. We present a systematic empirical evaluation of faithfulness in retrieval-augmented generation (RAG) systems using two benchmark datasets, HotpotQA and HaluBench, covering both multi-hop reasoning and single-hop hallucination detection. We analyze three small-to-mid-sized (2B-8B) open-weight LLMs in combination with multiple retrieval strategies, including sparse, dense, and hybrid approaches, as well as score-based and rank-based fusion techniques, enabling a comprehensive assessment of retrieval-generation interactions. By disentangling retrieval and generation errors, we characterize how different pipeline components contribute to hallucinations in RAG systems. Our analysis provides actionable insights and practical evaluation protocols, highlighting the critical role of robust retrieval and careful system design. These findings offer a benchmarking-oriented perspective for developing more reliable and faithful RAG systems within evaluated model scales.
The Abstraction and Reasoning Corpus (ARC) has become a prominent benchmark for assessing whether AI systems can perform human-like reasoning. Recent efforts on ARC have increasingly adopted program synthesis paradigms based on domain-specific languages (DSL). To tackle the combinatorial challenges of program synthesis, leading methods employ pretrained models to produce DSL program sequences and apply efficient search algorithms during inference. Monte Carlo Tree Search (MCTS) stands out among these, as it enables effective navigation of vast search spaces while maintaining a principled equilibrium between exploration and exploitation. In this work, we introduce a neural MCTS algorithm customized for ARC-AGI, which amplifies exploration by directing the search toward promising solutions via learned neural guidance, yielding robust performance on demanding ARC tasks. However, pretrained models frequently underperform on certain tasks owing to insufficient exposure to particular DSL tokens or structures during pretraining. To mitigate this, we additionally propose an LLM-guided technique, in which an LLM recommends pertinent tokens to prune the search space, thereby allowing the solver to prioritize previously overlooked DSL elements. Integrating the o4-mini model with our approach, we successfully solve nearly all DSL-solvable tasks in the ARC-AGI-1 evaluation benchmark, highlighting the effectiveness of our neural MCTS framework guided by LLM-based token recommendations.
Learning-based approaches have made substantial progress on solving combinatorial optimization problems, increasingly rivaling classical operations research methods. In particular, improvement-based machine learning methods, which iteratively refine an existing solution, have achieved state-of-the-art results on routing problems such as the traveling salesperson problem and the vehicle routing problem. Despite this success, analogous learning-based improvement methods for scheduling remain largely unexplored. To close this gap, we introduce a learning-based improvement method for scheduling based on the neural deconstruction framework, which improves solutions by iteratively applying a learned deconstruction policy followed by a simple repair strategy. We apply our method to both the classical and flexible job-shop scheduling problems. Our experimental results demonstrate that our method is able to outperform existing end-to-end and learning-augmented approaches on various well-known benchmark instances from the operations research literature.
Multiple-source domain adaptation (MDA), aims at transferring knowledge from several source domains to a little or no labeled target domain, has been widely and successfully applied in many fields of machine learning and image processing. Although many MDA algorithms have been proposed in recent years, there have been few theoretical research results. In this paper, we dedicate to studying theory and algorithm for MDA. We provide a theoretical analysis framework for multiple-source domain adaptation based on covering numbers in statistical learning theory. Under this framework, we comprehensively analyze the generalization error bounds of multiple-source domain adaptation in various situations, e.g. unsupervised MDA and semi-supervised MDA. Due to the large number of parameters involved, we also provide an efficient method for selecting parameters based on the derived generalization error bounds. Experimental results confirmed the effectiveness of the proposed method. Furthermore, we propose an adversarial multiple feature spaces adaptation network(AMFSAN) for MDA according to the theoretical results. Numerical simulations indicate that the proposed algorithm is superior to other existing algorithms.
Image captioning systems based on large vision–language models often rely on computationally intensive cross-attention architectures, limiting their suitability for real-time and resource-constrained deployment. In this work, we propose Photon, a lightweight prefix-conditioned multimodal captioning framework that combines a frozen MobileCLIP vision encoder with a compact decoder-only Transformer incorporating Rotary Positional Embeddings, RMSNorm, and SwiGLU activations. Visual information is injected through a small set of learned prefix tokens, enabling efficient multimodal conditioning without region-based processing or heavy cross-modal attention. On the MS-COCO Karpathy split, Photon achieves a CIDEr score of 108.59 while requiring only 12.41 M trainable parameters and 3.72 GFLOPs for image to caption generation. The model demonstrates competitive semantic alignment performance and improves inference efficiency, achieving 2.41 × GPU and 8.28 × CPU speed-ups compared to larger pretrained baselines. Zero-shot evaluation on Flickr30K, NoCaps and TextCaps further indicates consistent cross-dataset generalization across lexical, consensus-based, and embedding-based metrics. Batch-scaling analysis reveals near-linear throughput growth up to batch size 512, highlighting effective parallelization of the decoder. These results suggest that prefix-based multimodal conditioning with a modern lightweight decoder provides a favorable balance between caption quality and computational efficiency, making it suitable for practical deployment scenarios.
In semantic segmentation, even state-of-the-art deep learning models fall short of the performance required in certain high-stakes applications such as medical image analysis. In these cases, performance can be improved by allowing a model to abstain from making predictions when confidence is low, an approach known as selective prediction. While well-known in the classification literature, selective prediction has been underexplored in the context of semantic segmentation. This paper tackles the problem by focusing on image-level abstention, which involves producing a single confidence estimate for the entire image, in contrast to previous approaches that focus on pixel-level uncertainty. Assuming the Dice coefficient as the evaluation metric for segmentation, two main contributions are provided in this paper: (i) In the case of known marginal posterior probabilities, we derive the optimal confidence estimator, which is observed to be intractable for typical image sizes. Then, an approximation computable in linear time, named Soft Dice Confidence (SDC), is proposed and proven to be tightly bounded to the optimal estimator. (ii) When only an estimate of the marginal posterior probabilities are known, we propose a plug-in version of the SDC and show it outperforms all previous methods, including those requiring additional tuning data. These findings are supported by experimental results on both synthetic data and real-world data from six medical imaging tasks, including out-of-distribution scenarios, positioning the SDC as a reliable and efficient tool for selective prediction in semantic segmentation.
Simulating physical systems is essential in engineering, but analytical solutions are limited to straightforward problems. Consequently, numerical methods like the Finite Element Method (FEM) are widely used. However, the FEM becomes computationally expensive as problem complexity and accuracy demands increase. Adaptive Mesh Refinement (AMR) improves the FEM by dynamically placing mesh elements on the domain, balancing computational speed and accuracy. Classical AMR depends on heuristics or expensive error estimators, which may lead to suboptimal performance for complex simulations. While AMR methods based on machine learning are promising, they currently only scale to simple problems. In this work, we formulate AMR as a system of collaborating, homogeneous agents that iteratively split into multiple new agents. This agent-wise perspective enables a spatial reward formulation focused on reducing the maximum mesh element error. Our approach, Adaptive Swarm Mesh Refinement++ (ASMR++), offers efficient, stable optimization and generates highly adaptive meshes at user-defined resolution at inference time. Extensive experiments demonstrate that ASMR++ outperforms heuristic approaches and learned baselines, matching the performance of expensive error-based oracle AMR strategies. ASMR++ additionally generalizes to different domains during inference, and produces meshes that simulate up to 2 orders of magnitude faster than uniform refinements in more demanding settings.
