In this paper, the multi-agent patrolling problem subject to energy constraints is studied. Several existing online, distributed, heuristic-based patrolling algorithms are adapted to accommodate energy-constrained multi-agent patrolling. These adapted algorithms are evaluated on graphs using simulations in order to identify algorithm characteristics that are beneficial for energy-constrained multi-agent patrolling and to analyse the performance of the algorithms when subject to energy constraints. The experimental results also reveal several important considerations when planning multi-agent patrolling missions subject to energy constraints. Patrolling performance is measured using metrics based on idleness, which represents the time since a location within a patrolling area was previously visited. It was evident from the experimental results that coordination between agents, such as information sharing and/or conflict resolution, improves performance in general, especially as the number of agents increases. However, in the problem setting considered, sharing visit information alone appears to be detrimental to patrolling performance. Furthermore, careful consideration may be required regarding the patrolling area in order to ensure that agents subject to energy constraints can reach all locations when using certain distributed, heuristic-based patrolling algorithms. In addition, increasing the amount of energy available can negatively impact the performance of distributed, heuristic-based patrolling algorithms on some graphs. Overall, no single algorithm consistently outperformed all others across all graph topologies and different numbers of agents with respect to the evaluated patrolling metrics. This highlights the importance of considering each factor when planning distributed, heuristic-based multi-agent patrolling strategies under energy constraints.
Multi-agent reinforcement learning (MARL) has achieved notable progress in cooperative tasks, yet the common use of parameter sharing often leads to behavioral homogeneity, which restricts the diversity and adaptability of agent coordination. To address this issue, we propose Contrastive Learning for Role Representation (CLRE), a framework designed to promote strategic diversity by encouraging agents to learn distinct and semantically meaningful roles. The key idea is to employ a Large Language Model (LLM) as an offline domain expert that automatically generates semantic role labels from agent states. These labels provide reliable supervision for a contrastive learning objective, guiding the model to acquire role embeddings that are both discriminative and interpretable. To ensure these role representations effectively support coordination, CLRE incorporates a graph attention network that captures inter-agent relationships and a dual-attention value function that integrates role information with the global state for context-aware decision-making. Experiments on the StarCraft Multi-Agent Challenge (SMAC) benchmark demonstrate that CLRE achieves superior win rates and sample efficiency compared with state-of-the-art MARL methods. Ablation studies further confirm that each component contributes to the overall performance, showing that the proposed approach enables diverse and efficient cooperation among agents in complex environments.
This paper introduces a novel approach to advancing distributed cooperative learning in multi-agent systems (MASs) using Gaussian Process (GP) regression. The primary contribution lies in the development of a prior-aware selective distributed GPs (Pri-GP) algorithm, which allows agents to selectively collaborate by requesting predictions from neighboring agents based on their trustworthiness. This selective learning mechanism ensures that agents can avoid relying on neighbors with potentially flawed prior knowledge, thereby improving prediction accuracy, particularly in scenarios where an agent’s prior information is incorrect or outdated. Moreover, Pri-GP eliminates the need for costly variance calculations typically required for determining aggregation weights. We also extend this framework to the online-Pri-GP algorithm, which integrates real-time learning strategies, demonstrating superior performance in dynamical systems by achieving lower prediction errors. Additionally, we provide a prediction error bound within the Pri-GP framework, offering a guarantee of the aggregated predictions. Simulation results demonstrate that Pri-GP and online-Pri-GP outperform existing distributed GPs approaches in function approximation and system identification.
For cover function games (CoFGs), we address the problem of organising agents into possibly overlapping coalitions so as to maximise social welfare. We introduce a subclass of CoFGs called probabilistically monotone CoFGs. Monotonicity means that the closer a cover is to an optimum, the higher its welfare. Probabilistic monotonicity means that monotonicity is satisfied with some probability, i.e., some violations of monotonicity are permitted as long as the number of violations is bounded. In addition, externalities are permitted and social welfare is not restricted to the sum function. For such games, we obtain a bound on the number of monotonicity violations that can be permitted for computing an exact optimum. For probabilistically monotone CoFGs with a bound on the number of monotonicity violations, we devise algorithms for computing an exact optimum and analyze their time complexities. We also provide constructive proofs which form the basis for our algorithms. Placing our algorithms in the context of the existing literature, we note that a key unique feature of our algorithms is that they do not require the numeric welfare values of covers as input, rather, only an ordering over these values is required. Moreover, the ordering that is required as input does not have to be the actual ordering over the welfare values, it can be an approximate anticipated version of the actual ordering. The anticipated ordering is allowed to differ from the actual ordering in that anticipation errors, captured by monotonicity violations, are permitted in the anticipated ordering as long as the number of errors is bounded. Our algorithms are therefore highly relevant for practical applications like multi-agent task allocation where welfare values are only revealed after coalition formation although an ordering over the values can be approximately anticipated in advance of coalition formation.
