Activation steering can change behaviour without establishing that the effect is specific to the intended concept. We introduce SteerCheck, a preregistered attribution audit that matches off-target KL and separates mean, protected-tail, polarity, transfer, and semantic claims. Exact replay of 960 Qwen3-14B interventions reveals complementary limits of common controls: isotropic directions occupy a narrow near-orthogonal region, whereas sign-randomized same-construction directions often retain substantial target alignment. Effect is strongly associated with signed cosine within the sign-randomized family (ρ=.94); 25.3% of its draws exceed cosine .5, and every draw exceeding the observed mean effect has cosine above .80. This alignment leakage does not by itself invalidate a conditional randomization test; it limits what the comparator can distinguish and motivates reporting exchangeability assumptions, a construction diagnostic A, and the empirical cosine distribution. The primary Qwen complete gate remains negative because the protected tail fails all families. On independent data, continuous margin transfers only in Qwen and accuracy transfers in no selected cell. Prospectively registered language controls pass the complete gate in Qwen and DeepSeek, while a passing DeepSeek detox comparator rules out categorical separation; all nominal passes are sensitive to Γ=1.10. Frozen three-rater open-generation evaluation supports factual correction in DeepSeek but not Qwen; the automatic judge fails calibration (macro-F1 .562), so null-wide semantic results remain descriptive. SteerCheck makes these conditional and mixed conclusions auditable.

Skill discovery has emerged as a popular route for unsupervised reinforcement learning (URL), offering agents a diverse, reusable set of behaviours learned before any task-specific reward is experienced. However, existing methodologies tend to favour either categorical codes or unimodal skill priors, which simplifies training at the cost of limiting the variety of behaviours they can represent. We introduce Discovery of Mixture Skills (DiMS), a URL algorithm that learns a latent Gaussian mixture by training a Gaussian Mixture Variational Autoencoder (GMVAE) in tandem with the unsupervised policy. In DiMS, a hierarchical GMVAE simultaneously discovers clusters of skills, while an auxiliary macro-latent dynamically positions mixture components to prevent mode collapse. A joint loss term combining log-likelihood and curiosity rewards enables simultaneous updates of representation and policy while improving exploration. Experiments on the Unsupervised Reinforcement Learning Benchmark (URLB) show that DiMS consistently outperforms a wide range of state-of-the-art baselines. Ablation studies confirm that the mixture prior is critical to these gains, and that DiMS is robust to alternative exploration bonuses. Overall, our results suggest that Gaussian mixture skill priors offer a compelling foundation for future unsupervised RL.

Coordination graphs are a central abstraction in cooperative multi-agent reinforcement learning (MARL), yet existing sparse-graph learners lack a theoretically grounded mechanism to decide which edges should exist and how much information each edge should carry. Current methods rely on heuristic criteria that offer no formal guarantee on the learned topology, and no principled way to allocate different communication capacities to structurally different agent relationships. To address this, we propose Heterogeneous Information-Bottleneck Coordination Graphs (HIBCG), which learns a group-aware sparse graph in which both edge existence and message capacity are theoretically justified. With the graph information bottleneck (GIB) serving as the underlying tool, HIBCG first constructs a group-aligned block-diagonal prior that provides a closed-form criterion for edge retention – determining which edges should exist and at what density per group block – and then controls per-agent feature bandwidth on the resulting topology, compressing messages to retain only task-relevant content. We prove that the group-aligned prior strictly tightens the variational bound on topology learning, that the objective decomposes per group block, enabling differential edge control, and that capacity allocation follows a water-filling principle.

KV-cache compression methods are predominantly evaluated with the query appended to the context before compression – a query-aware protocol. Yet the economic case for a compressed KV cache is reuse: compress a document once, answer many future questions against it. In that deployment, compression must happen query-agnostic – before any question is seen. We present a matched-budget audit of six published compression methods against three trivial baselines on three open 7-9B models (144,300 paired evaluations on RULER-8192; 40,800 on LongBench; 50,000-resample paired bootstrap throughout). Everything is held fixed – model, compression ratio, instances, decoding – except the scoring rule. Three findings. (1) Query visibility changes the rankings: under the agnostic protocol, of the five audited methods that share a common attention backend, only KeyDiff beats a best-of-3 trivial baseline consistently (31 of 36 cells), and the most widely deployed method, SnapKV, loses to "keep the start and the recent window" on average (-0.066). (2) The per-method drop between the two protocols is ordered consistently with how visible the question is to each method's scoring signal, legible in its source code: from Delta=+0.198 for SnapKV (the question sits inside its 64-token observation window) down to Delta=+0.011 for KeyDiff (its score contains no query term at all).

Vision-Language-Action (VLA) models bridge multimodal reasoning with physical control, but adapting them to new tasks with scarce demonstrations remains unreliable. While fine-tuned VLA policies often produce semantically plausible trajectories, failures often arise from unresolved geometric ambiguities, where near-miss actions lead to divergent execution outcomes under limited supervision. We study few-shot VLA adaptation from a generation-selection perspective and propose a novel framework, VGAS (Value-Guided Action-chunk Selection). It performs inference-time best-of-N selection to identify action chunks that are both semantically faithful and geometrically precise. Specifically, VGAS employs a finetuned VLA as a high-recall proposal generator and introduces Q-Chunk-Former, a geometrically grounded Transformer critic to resolve fine-grained geometric ambiguities. In addition, we propose Explicit Geometric Regularization (EGR), which shapes a discriminative value landscape to preserve action ranking resolution among near-miss candidates while mitigating value instability under scarce supervision. Experiments and theoretical analysis demonstrate that VGAS consistently improves success rates and robustness under limited demonstrations and distribution shifts. Our code is available at https://github.com/Jyugo-15/VGAS

Model-based reinforcement learning (MBRL) aims to promote sample efficiency and reduce the number of interactions with the true environment, via learning an environment dynamic model, compared with model-free reinforcement learning (MFRL). However, the success of MBRL heavily relies on two key aspects: model learning and planning. The former refers to learning an accurate model, and the latter aims to improve the behavior policy. In this article, we investigate these two aspects further with model ensemble learning. We design a deep residual attention U-Net (RauNet) with fewer neurons (or weights) than the widely used shallow neural network as our base models and further apply the Hilbert-Schmidt independence criterion (HSIC) as a regularization term to pursue model diversity explicitly for the model ensemble. Furthermore, we propose an adaptive trust region policy optimization (TRPO), in which the parametric Renyi alpha divergence substitutes for the Kullback-Leibler (KL) divergence for measuring the difference between two successive policies, and the alpha value can be adaptively adjusted during TRPO training iterations. This method is called diversity-driven model ensemble adaptive TRPO, or simply diversity-driven model ensemble adaptive trust region policy optimization. Our detailed experiments on six benchmark environments show that our proposed approach is optimal, compared with five state-of-the-art RL techniques.

Autonomous drone navigation with deep reinforcement learning (DRL) is hindered by the difficulty of specifying reward functions for vision-based, continuous control in complex environments. We address this by using inverse reinforcement learning (IRL) to recover a task-aligned reward directly from expert demonstrations; specifically, we employ adversarial IRL (AIRL) to learn the reward. In evaluation, the learned-reward policy improves success rate, smoothness and trajectories consistency compared with a carefully tuned human-designed reward and baselines reward function. These results indicate that learning the reward from demonstrations provides a precise and transferable objective for autonomous flight, achieving better performance and better guidance under verification of our evaluation protocol without manual reward engineering. To the best of our knowledge, this is the first work to successfully apply an AIRL framework for visual drone navigation.

Clustering is a fundamental task in machine learning and data mining. The success of deep learning, especially deep generative models, has given birth to the next generation of clustering - deep unsupervised clustering featured by the capability of clustering large-scale and high-dimensional data. One typical example is the Gaussian mixture variational autoencoder (GMVAE), where the prior of the latent variable is a mixture of Gaussian distributions (GMM). However, the mixture number manually given in GMVAE greatly impacts on the final clustering performance and also limits the adaptation ability when new clusters come. To solve this issue, we propose an Infinite Gaussian Mixture Autoencoder (IGMVAE), which replaces the plain Gaussian prior in VAE with an infinite Gaussian mixture distribution. Our method adaptively learns the number of mixtures from data, overcoming the manual setting limitation in GMVAE, and dynamically expands clusters for novel categories. Experimental results on three datasets show that our proposed IGMVAE achieves 5.5%–14.25% higher clustering accuracy compared to GMVAE. Moreover, our model is capable of generating high-quality data in a fully unsupervised manner.

Self-Regulated Learning (SRL) enhances outcomes but remains challenging to support in digital environments due to the complexity of real-time evaluation. While Reinforcement Learning (RL) enables personalised scaffolding based on students' SRL strategies, it requires modelling per student, making it impractical for real-time settings. We study Transfer Reinforcement Learning (TRL) as a scalable framework, enabling transfer of optimal strategies across contexts defined by learning outcomes. Using data from 92 secondary school students completing a writing task, we analysed strategies of those with the highest scores and gains to assess transfer feasibility. Successor Representations were implemented to evaluate the impact of leveraging prior SRL strategies. Results reveal notable similarities across contexts, indicating potential for reuse. Moreover, Successor Representations enabled effective knowledge transfer, outperforming classical RL and eliminating retraining. These findings demonstrate TRL's potential for scalable, personalised SRL support.

Artificial Intelligence (AI) has emerged as a transformative tool in precision agriculture, facilitating datadriven decision-making and crop improvement. In the context of agricultural crops, data from multiple modalities, such as phenotypic traits, genomic markers, and environmental conditions, offer diverse insights into crop development and yield potential. However, single-modality approaches may fail to capture the complex interplay between genomics, environment, and other factors affecting crop traits. To address this challenge, this study investigates the integration of multimodal data to improve genotype-to-phenotype predictions. Focusing on barley (Hordeum vulgare L.), a globally and nationally important cereal crop, we propose a new barley-Multimodal Deep Learning (barley-MMDL) model to predict flowering time and grain yield using heterogeneous multimodal datasets. The model combines Convolutional Neural Networks (CNNs) to process high-dimensional genomic markers with Long Short-Term Memory (LSTM) networks to capture temporal patterns in environmental data. These modality-specific latent features are then fused to enable joint optimization of feature extraction and prediction in an end-to-end manner. The proposed barleyMMDL model achieved the lowest RMSE values of 8.84 for flowering time and 778.50 for grain yield, outperforming baseline unimodal and multimodal models. These results demonstrate the improved predictive capability of barleyMMDL and underscore the potential of multimodal data integration to advance prediction capability in precision agriculture and contribute to sustainable agricultural practices.

Model-based reinforcement learning (RL) aims to learn the underlying dynamics of a given environment. The success of most existing works is built on the critical assumption that the dynamic is fixed, which is unrealistic in many open-world scenarios, such as drone delivery and online chatting, where agents may need to deal with environments with unpredictable changing dynamics (hereafter, real non-stationary environment). Therefore, learning changing dynamics in a real non-stationary environment offers both significant benefits and challenges. This paper proposes a new model-based reinforcement learning algorithm that proactively and dynamically detects possible changes and Learns these Latent and Changing Dynamics (LLCD) in a latent Markovian space for real non-stationary environments. To ensure the Markovian property of the RL model and improve computational efficiency, we employ a latent space model to learn the environment's transition dynamics. Furthermore, we perform online change detection in the latent space to promptly identify change points in non-stationary environments. Then, we utilize the detected information to help the agent adapt to new conditions. Experiments indicate that the rewards of the proposed algorithm accumulate for the most rapid adaptions to environmental change, among other benefits. This work has a strong potential to enhance environmentally suitable model-based reinforcement learning capabilities.

Traditional parameter-space posterior inference for Bayesian neural networks faces several challenges, such as the difficulty in specifying meaningful prior, the potential pathologies in deep models and the intractability for multi-modal posterior. To address these issues, functional variational inference (fVI) and functional Markov Chain Monte Carlo (fMCMC) are two recently emerged Bayesian inference schemes that perform posterior inference directly in function space by incorporating more informative functional priors. Similar to their parameter-space counterparts, fVI and fMCMC have their own strengths and weaknesses. For instance, fVI is computationally efficient but imposes strong distributional assumptions, while fMCMC is asymptotically exact but suffers from slow mixing in high dimensions. To inherit the complementary benefits of both schemes, this work proposes a novel hybrid inference method for functional posterior inference. Specifically, it combines fVI and fMCMC successively by an elaborate linking mechanism to form an alternating approximation process. We also provide theoretical justification for the soundness of such a hybrid inference through the lens of Wasserstein gradient flows in the function space. We evaluate our method on several benchmark tasks and observe improvements in both predictive accuracy and uncertainty quantification compared to parameter/function-space VI and MCMC.

Graph-based multi-agent reinforcement learning (MARL) enables coordinated behavior under partial observability by modeling agents as nodes and communication links as edges. While recent methods excel at learning sparse coordination graphs-determining who communicates with whom-they do not address what information should be transmitted under hard bandwidth constraints. We study this bandwidth-limited regime and show that naive dimensionality reduction consistently degrades coordination performance. Hard bandwidth constraints force selective encoding, but deterministic projections lack mechanisms to control how compression occurs. We introduce Bandwidth-constrained Variational Message Encoding (BVME), a lightweight module that treats messages as samples from learned Gaussian posteriors regularized via KL divergence to an uninformative prior. BVME's variational framework provides principled, tunable control over compression strength through interpretable hyperparameters, directly constraining the representations used for decision-making. Across SMACv1, SMACv2, and MPE benchmarks, BVME achieves comparable or superior performance while using 67–83

Classical parameter-space Bayesian inference for Bayesian neural networks (BNNs) suffers from several unresolved prior issues, such as knowledge encoding intractability and pathological behaviours in deep networks, which can lead to improper posterior inference. To address these issues, functional Bayesian inference has recently been proposed leveraging functional priors, such as the emerging functional variational inference. In addition to variational methods, stochastic gradient Markov Chain Monte Carlo (MCMC) is another scalable and effective inference method for BNNs to asymptotically generate samples from the true posterior by simulating continuous dynamics. However, existing MCMC methods perform solely in parameter space and inherit the unresolved prior issues, while extending these dynamics to function space is a non -trivial undertaking. In this paper, we introduce novel functional MCMC schemes, including stochastic gradient versions, based on newly designed diffusion dynamics that can incorporate more informative functional priors. Moreover, we prove that the stationary measure of these functional dynamics is the target posterior over functions. Our functional MCMC schemes demonstrate improved performance in both predictive accuracy and uncertainty quantification on several tasks compared to naive parameter-space MCMC and functional variational inference.

Offline-to-online reinforcement learning (RL) leverages both pre-trained offline policies and online policies trained for downstream tasks, aiming to improve data efficiency and accelerate performance enhancement. An existing approach, Policy Expansion (PEX), utilizes a policy set composed of both policies without modifying the offline policy for exploration and learning. However, this approach fails to ensure sufficient learning of the online policy due to an excessive focus on exploration with both policies. Since the pre-trained offline policy can assist the online policy in exploiting a downstream task based on its prior experience, it should be executed effectively and tailored to the specific requirements of the downstream task. In contrast, the online policy, with its immature behavioral strategy, has the potential for exploration during the training phase. Therefore, our research focuses on harmonizing the advantages of the offline policy, termed exploitation, with those of the online policy, referred to as exploration, without modifying the offline policy. In this study, we propose an innovative offline-to-online RL method that employs a non-monolithic exploration approach. Our methodology demonstrates superior performance compared to PEX. The code for this comparison is readily available. Source-Code .

Transfer reinforcement learning (TRL) aims to build a policy for a unseen target task by reusing previously found optimal policies from a fixed set of source tasks. Successor features (SFs) and Generalised Policy Improvement (GPI) have been adopted as a robust framework to achieve TRL expressing rewards as a linear combination of a shared feature function and a task-specific vector weights. SF-GPI are used for reusing, fast learning, and composing a fixed set of policies derived from SFs through two-level decision-making, which can be understood as a hierarchical method. However, SF-GPI performance on target tasks is related to the distance from the source tasks’ vector weights, limiting their ability to ensure optimal policies for more distant tasks. In this paper, we introduce a novel composition mechanism by defining a new Markov decision process involving high-level states and actions constructed of SFs, resulting in a high-level policy which adaptively selects high-level actions. This approach facilitates knowledge transfer across a varied-distance set of tasks without performance degradation. Finally, we empirically demonstrate that our approach learns and composes SF representations, outperforming SF-GPI by maintaining the performance across a varied-distance set of tasks in 3 widely-used problems in the SFs literature.

Effective agent coordination is crucial in cooperative multiagent reinforcement learning (MARL). While agent cooperation can be represented by graph structures, prevailing graph learning methods in MARL are limited. They rely solely on one-step observations, neglecting crucial historical experiences, leading to deficient graphs that foster redundant or detrimental information exchanges. In addition, high computational demands for action-pair calculations in dense graphs impede scalability. To address these challenges, we propose inferring a latent temporal sparse coordination graph (LTS-CG) for MARL. The LTS-CG leverages agents' historical observations to calculate an agent-pair probability matrix, where a sparse graph is sampled from and used for knowledge exchange between agents, thereby simultaneously capturing agent dependencies and relationship uncertainty. The computational complexity of this procedure is only related to the number of agents. This graph learning process is further augmented by two innovative characteristics: Predict-Future, which enables agents to foresee upcoming observations, and Infer-Present, ensuring a thorough grasp of the environmental context from limited data. These features allow LTS-CG to construct temporal graphs from historical and real-time information, promoting knowledge exchange during policy learning and effective collaboration. Graph learning and agent training occur simultaneously in an end-to-end manner. Our demonstrated results on the StarCraft II benchmark underscore LTS-CG's superior performance.

Meta-reinforcement learning (meta-RL) algorithms extract task information from experienced context in order to reason about new tasks, and facilitate rapid adaptation. The quality of these contextual representations (or embeddings) is therefore crucial for a meta-RL agent to make effective decisions in unknown environments. Current methods predominantly assume the existence of a single underlying task, but using a single contextual embedding may not be expressive enough to fully capture the broader distribution of task variations that an agent might encounter. Decomposing that information into different representations can allow them to capture more relevant features in context space while applying additional structure that aids downstream exploitation. In this article, we develop global-local embeddings for contextual meta-RL (GLOBEX), an off-policy contextual meta-RL algorithm that decomposes the contextual representation into separate global and local embeddings. The learning process maximizes information retained by the embeddings and utilizes a mutual information constraint to encourage decoupling. Illustrative examples show that our method effectively adapts by identifying global task dynamics and exploiting temporally local signals. In addition, GLOBEX outperforms existing state-of-the-art meta-RL algorithms on standard MuJoCo benchmarks.

Trust region policy optimization (TRPO) is an iterative reinforcement learning algorithm that both maximizes a surrogate objective based on a generalized advantage function and enforces a trust region constraint between two consecutive policies based on the Kullback-Leibler divergence in each iteration. On the one hand, its surrogate objective only approximates the positive theoretical improvement and thus relaxes the theoretical guarantee. One the other hand, currently, TRPO is approximately solved via a linear approximation for its objective and a quadratic approximation for its constraint, which results in an upper bound for the linear search of the step size. However, this solution strategy does not provide a lower bound for the step size search. Intuitively, a lower bound could be determined manually, but such a setting has no physical meaning. To this end, we present an alternative solution in this article, which is to exchange the objective and the constraint in TRPO according to reciprocal optimization technique, and then derive a reciprocal TRPO. Applying a similar approximation solution of TRPO to our reciprocal TRPO produces a lower bound for the step size search that has a physical interpretation for improving the surrogate objective or return. Further, we aggregate the original TRPO with this reciprocal TRPO to construct a twin TRPO, whose step size has a lower bound from our reciprocal TRPO and an upper bound from TRPO, to facilitate the policy optimization and achieve a least return improvement. Extensive experiments on twelve benchmark environments show that our twin TRPO is superior to six existing techniques: the original TRPO, proximal policy optimization, off-policy TRPO and two entropy regularized TRPOs, as well as our reciprocal TRPO. Our code is available at https://github.com/HTXu-UTS/twinTRPO.

In networked multi-agent reinforcement learning (Networked-MARL), decentralized agents must act autonomously under local observability and constrained communication over fixed physical graphs. Existing methods often assume static neighborhoods, limiting adaptability to dynamic or heterogeneous environments. While centralized frameworks can learn dynamic graphs, their reliance on global state access and centralized infrastructure is impractical in real-world decentralized systems. We propose a stochastic graph-based policy for Networked-MARL, where each agent conditions its decision on a sampled subgraph over its local physical neighborhood. Building on this formulation, we introduce \textbf{BayesG}, a decentralized actor–critic framework that learns sparse, context-aware interaction structures via Bayesian variational inference. Each agent operates over an ego-graph and samples a latent communication mask to guide message passing and policy computation. The variational distribution is trained end-to-end alongside the policy using an evidence lower bound (ELBO) objective, enabling agents to jointly learn both interaction topology and decision-making strategies. BayesG outperforms strong MARL baselines on large-scale traffic control tasks with up to 167 agents, demonstrating superior scalability, efficiency, and performance.