Schelling's model considers k types of agents each of whom needs to select a vertex on an undirected graph, where every agent prefers neighboring agents of the same type. We are motivated by a recent line of work that studies solutions that are optimal with respect to notions related to the welfare of the agents. We explore the parameterized complexity of computing such solutions. We focus on the well-studied notions of social welfare (WO) and Pareto optimality (PO), alongside the recently proposed notions of group-welfare optimality (GWO) and utility-vector optimality (UVO), both of which lie between WO and ¶O. Firstly, we focus on the fundamental case where k=2 and there are r red agents and b blue agents. We show that all solution-notions we consider are intractable even when b=1 and that they do not admit an FPT algorithm when parameterized by r and b, unless FPT = W[1]. In addition, we show that WO and GWO remain intractable even on cubic graphs. We complement these negative results with an FPT algorithm parameterized by r, b and the maximum degree of the graph. For the general case with k types of agents, we prove that for any of the notions we consider the problem remains hard when parameterized by k for a large family of graphs that includes trees. We accompany these negative results with an XP algorithm parameterized by k and the treewidth of the graph.
Imitation Learning (IL) algorithms such as behavior cloning are a promising direction for learning human-level driving behavior. However, these approaches do not explicitly infer the underlying causal structure of the learned task. This often leads to misattribution about the relative importance of scene elements towards the occurrence of a corresponding action, a phenomenon termed causal confusion or causal misattribution. Causal confusion is made worse in highly complex scenarios such as urban driving, where the agent has access to a large amount of information per time step (visual data, sensor data, odometry, etc.). Our key idea is that while driving, human drivers naturally exhibit an easily obtained, continuous signal that is highly correlated with causal elements of the state space: eye gaze. We collect human driver demonstrations in a CARLA-based VR driving simulator, DReyeVR, allowing us to capture eye gaze in the same simulation environment commonly used in prior work. Further, we propose a contrastive learning method to use gaze-based supervision to mitigate causal confusion in driving IL agents — exploiting the relative importance of gazed-at and not-gazed-at scene elements for driving decision-making. We present quantitative results demonstrating the promise of gaze-based supervision improving the driving performance of IL agents.
To overcome incompatibility issues, kidney patients may swap their donors. In international kidney exchange programmes (IKEPs), countries merge their national patient-donor pools. We consider a recently introduced credit system. In each round, countries are given an initial "fair" allocation of the total number of kidney transplants. This allocation is adjusted by a credit function yielding a target allocation. The goal is to find a solution that approaches the target allocation as closely as possible, to ensure long-term stability of the international pool. As solutions, we use maximum matchings that lexicographically minimize the country deviations from the target allocation. We perform, for the first time, a computational study for a large number of countries. For the initial allocations we use two easy-to-compute solution concepts, the benefit value and the contribution value, and four classical but hard-to-compute concepts, the Shapley value, nucleolus, Banzhaf value and tau value. By using state-of-the-art software we show that the latter four concepts are now within reach for IKEPs of up to fifteen countries. Our experiments show that using lexicographically minimal maximum matchings instead of ones that only minimize the largest deviation from the target allocation (as previously done) may make an IKEP up to 54% more balanced.
In this paper, we investigate under which conditions normal-form games are (guaranteed) to be strategically equivalent. First, we show for N-player games (N >= 3) that (A) it is NP-hard to decide whether a given strategy is a best response to some strategy profile of the opponents, and that (B) it is co-NP-hard to decide whether two games have the same best-response sets. Combining that with known results from the literature, we move our attention to equivalence-preserving game transformations. It is a widely used fact that a positive affine (linear) transformation of the utility payoffs neither changes the best-response sets nor the Nash equilibrium set. We investigate which other game transformations also possess either of the following two properties when being applied to an arbitrary N-player game (N >= 2): (i) The Nash equilibrium set stays the same; (ii) The best-response sets stay the same. For game transformations that operate player-wise and strategy-wise, we prove that (i) implies (ii) and that transformations with property (ii) must be positive affine. The resulting equivalence chain highlights the special status of positive affine transformations among all the transformation procedures that preserve key game-theoretic characteristics.
Nash Equilibrium (NE) is the canonical solution concept of game theory, which provides an elegant tool to understand the rationalities. Though mixed strategy NE exists in any game with finite players and actions, computing NE in two- or multi-player general-sum games is PPAD-Complete. Various alternative solutions, e.g., Correlated Equilibrium (CE), and learning methods, e.g., fictitious play (FP), are proposed to approximate NE. For convenience, we call these methods as ``inexact solvers'', or ``solvers'' for short. However, the alternative solutions differ from NE and the learning methods generally fail to converge to NE. Therefore, in this work, we propose REinforcement Nash Equilibrium Solver (RENES), which trains a single policy to modify the games with different sizes and applies the solvers on the modified games where the obtained solution is evaluated on the original games. Specifically, our contributions are threefold. i) We represent the games as alpha-rank response graphs and leverage graph neural network (GNN) to handle the games with different sizes as inputs; ii) We use tensor decomposition, e.g., canonical polyadic (CP), to make the dimension of modifying actions fixed for games with different sizes; iii) We train the modifying strategy for games with the widely-used proximal policy optimization (PPO) and apply the solvers to solve the modified games, where the obtained solution is evaluated on original games. Extensive experiments on large-scale normal-form games show that our method can further improve the approximation of NE of different solvers, i.e., alpha-rank, CE, FP and PRD, and can be generalized to unseen games.
Reinforcement Learning (RL) is a widely employed technique in decision-making problems, encompassing two fundamental operations -- policy evaluation and policy improvement. Enhancing learning efficiency remains a key challenge in RL, with many efforts focused on using ensemble critics to boost policy evaluation efficiency. However, when using multiple critics, the actor in the policy improvement process can obtain different gradients. Previous studies have combined these gradients without considering their disagreements. Therefore, optimizing the policy improvement process is crucial to enhance learning efficiency. This study focuses on investigating the impact of gradient disagreements caused by ensemble critics on policy improvement. We introduce the concept of uncertainty of gradient directions as a means to measure the disagreement among gradients utilized in the policy improvement process. Through measuring the disagreement among gradients, we find that transitions with lower uncertainty of gradient directions are more reliable in the policy improvement process. Building on this analysis, we propose a method called von Mises-Fisher Experience Resampling (vMFER), which optimizes the policy improvement process by resampling transitions and assigning higher confidence to transitions with lower uncertainty of gradient directions. Our experiments demonstrate that vMFER significantly outperforms the benchmark and is particularly well-suited for ensemble structures in RL.
We show how hidden interesting subelections can be discovered in ordinal elections. An interesting subelection consists of a reasonably large set of voters and a reasonably large set of candidates such that the former have a consistent opinion about the latter. Consistency may take various forms but we focus on three: Identity (all selected voters rank all selected candidates the same way), antagonism (half of the selected voters rank candidates in some order and the other half in the reverse order), and clones (all selected voters rank all selected candidates contiguously in the original election). We first study the computation of such hidden subelections. Second, we analyze synthetic and real-life data, and find that identifying hidden consistent subelections allows us to uncover some relevant concepts.
While reinforcement learning (RL) algorithms have been successfully applied to numerous tasks, their reliance on neural networks makes their behavior difficult to understand and trust. Counterfactual explanations are human-friendly explanations that offer users actionable advice on how to alter the model inputs to achieve the desired output from a black-box system. However, current approaches to generating counterfactuals in RL ignore the stochastic and sequential nature of RL tasks and can produce counterfactuals that are difficult to obtain or do not deliver the desired outcome. In this work, we propose RACCER, the first RL-specific approach to generating counterfactual explanations for the behavior of RL agents. We first propose and implement a set of RL-specific counterfactual properties that ensure easily reachable counterfactuals with highly probable desired outcomes. We use a heuristic tree search of the agent's execution trajectories to find the most suitable counterfactuals based on the defined properties. We evaluate RACCER in two tasks as well as conduct a user study to show that RL-specific counterfactuals help users better understand agents' behavior compared to the current state-of-the-art approaches.
We study strategic behavior of project proposers in the context of participatory budgeting. We assume that the votes are fixed and known and the proposers want to set as high project prices as possible, provided that their projects get selected and the prices are not below the minimum costs of their delivery. We study the existence of Nash equilibria in such games. Furthermore, we report an experimental study of the games we propose.
Diffusion auction design is a new trend in mechanism design for which the main goal is to incentivize existing buyers to invite new buyers, who are their neighbors on a social network, to join an auction even though they are competitors. With more buyers, a diffusion auction will be able to give a more efficient allocation and receive higher revenue. Existing studies have proposed many interesting diffusion auctions to attract more buyers, but the seller’s revenue is not optimized. Hence, in this study, we investigate what optimal revenue the seller can achieve by attracting more buyers. Different from the traditional setting, the revenue that can be achieved in a diffusion auction highly relies on the structure of the network. Hence, we focus on optimal auctions with given classes of underlying networks. We propose a class of mechanisms, where for any given structure, an optimal diffusion mechanism can be found. We point out that it implies an idea of “reserve structure”. Moreover, we show that an optimal mechanism that handles all structures does not exist. Therefore, we also propose mechanisms that have bounded approximations of the optimal revenue in all structures.
This paper presents a new research direction for online Multi-Level Aggregation (MLA) with delays. In this problem, we are given an edge-weighted rooted tree $T$, and we have to serve a sequence of requests arriving at its vertices in an online manner. Each request $r$ is characterized by two parameters: its arrival time $t(r)$ and location $l(r)$ (a vertex). Once a request $r$ arrives, we can either serve it immediately or postpone this action until any time $t > t(r)$. We can serve several pending requests at the same time, and the service cost of a service corresponds to the weight of the subtree that contains all the requests served and the root of $T$. Postponing the service of a request $r$ to time $t > t(r)$ generates an additional delay cost of $t - t(r)$. The goal is to serve all requests in an online manner such that the total cost (i.e., the total sum of service and delay costs) is minimized. The current best algorithm for this problem achieves a competitive ratio of $O(d^2)$ (Azar and Touitou, FOCS'19), where $d$ denotes the depth of the tree. Here, we consider a stochastic version of MLA where the requests follow a Poisson arrival process. We present a deterministic online algorithm which achieves a constant ratio of expectations, meaning that the ratio between the expected costs of the solution generated by our algorithm and the optimal offline solution is bounded by a constant. Our algorithm is obtained by carefully combining two strategies. In the first one, we plan periodic oblivious visits to the subset of frequent vertices, whereas in the second one, we greedily serve the pending requests in the remaining vertices. This problem is complex enough to demonstrate a very rare phenomenon that ``single-minded" or ``sample-average" strategies are not enough in stochastic optimization.
The field of emergent communication investigates the emergence of shared linguistic conventions among autonomous agents engaged in cooperative tasks that require communication. Conventions that arise through self-organisation are known to be more robust, flexible, and adaptive, and it removes the need for hand-crafting communication protocols. In my PhD research, I investigate how artificial agents can co-construct such conventions of linguistic structures in reference-based tasks. This problem is tackled using the language game experimental paradigm which aims to model the processes underlying the emergence and evolution of human languages. My primary contribution thus far introduces a novel methodology for the language game paradigm in the emergent setting. Using the methodology, agents can establish through self-organisation an emergent language that enables them to refer to arbitrary entities in their environment using single-word utterances. For the first time, the methodology is directly applicable to any dataset that describes entities in terms of continuously-valued features. The next phase in my research is to move from single-word utterances to multi-word utterances through the emergence of grammatical structures.
A wide range of real-world applications can be formulated as Multi-Agent Path Finding (MAPF) problem, where the goal is to find collision-free paths for multiple agents with individual start and goal locations. State-of-the-art MAPF solvers are mainly centralized and depend on global information, which limits their scalability and flexibility regarding changes or new maps that would require expensive replanning. Multi-agent reinforcement learning (MARL) offers an alternative way by learning decentralized policies that can generalize over a variety of maps. While there exist some prior works that attempt to connect both areas, the proposed techniques are heavily engineered and very complex due to the integration of many mechanisms that limit generality and are expensive to use. We argue that much simpler and general approaches are needed to bring the areas of MARL and MAPF closer together with significantly lower costs. In this paper, we propose Confidence-based Auto-Curriculum for Team Update Stability (CACTUS) as a lightweight MARL approach to MAPF. CACTUS defines a simple reverse curriculum scheme, where the goal of each agent is randomly placed within an allocation radius around the agent's start location. The allocation radius increases gradually as all agents improve, which is assessed by a confidence-based measure. We evaluate CACTUS in various maps of different sizes, obstacle densities, and numbers of agents. Our experiments demonstrate better performance and generalization capabilities than state-of-the-art MARL approaches with less than 600,000 trainable parameters, which is less than 5 of the neural network size of current MARL approaches to MAPF.
In the context of collaborative robotics, distributed situation awareness is essential for supporting collective intelligence in teams of robots and human agents, where it can be used for both individual and collective decision support. This is particularly important in applications pertaining to emergency rescue and crisis management. During operational missions, data and knowledge is gathered incrementally and in different ways by heterogeneous robots and humans. This paper aims to describe an RDF Graph Synchronization System called RGS⊕. It is assumed that a dynamic set of agents provide or retrieve knowledge stored in their local RDF Graphs which are continuously synchronized between agents. The RGS⊕ System was designed to handle unreliable communication and does not depend on a static centralized infrastructure.
In game theory, a fundamental class of games is impartial combinatorial games (ICGs). One of the challenging and long-standing problems of ICGs is to compute generalized winning strategies for possibly infinite number of legal states. Recently, Wu et al. proposed an automated method to synthesize generalized winning strategies of infinite-state ICGs. Their method has two major drawbacks: (1) it fails to generate winning formula with large size; and (2) it cannot usually construct the winning strategy even the winning formula is obtained. To tackle the above two drawbacks, in this paper, we propose the problem of exact binary classification and design a partial MaxSAT-based method to this problem. Then, we reduce the synthesis problem of generalized winning strategies of infinite-state ICGs to exact binary classification. The experimental results show that our method is more scalable and effective than Wu et al.'s approach.
In the last decade, social planners have used crowdfunding to raise funds for public projects. As these public projects are non-excludable, the beneficiaries may free-ride. Thus, there is a need to design incentive mechanisms for such strategic agents to contribute to the project. The existing mechanisms, like PPR or PPRx, assume that the agent's beliefs about the project getting funded do not change over time, i.e., their beliefs are static. Researchers highlight that unless appropriately incentivized, the agents defer their contributions in static settings, leading to a "race'' to contribute at the deadline. In this work, we model the evolution of agents' beliefs as a random walk. We study PPRx - an existing mechanism for the static belief setting - in this dynamic belief setting and refer to it as PPRx-DB for readability. We prove that in PPRx-DB, the project is funded at equilibrium. More significantly, we prove that under certain conditions on agent's belief evolution, agents will contribute as soon as they arrive at the mechanism. Thus, we believe that by incorporating dynamic belief evolution in analysis, the social planner may mitigate the concern of race conditions in many mechanisms.
We use multi-agent systems to solve conflicts between drone flight paths (4D contracts) in urban traffic, installing safety and service quality. Intuitive corrective actions are chosen considering delay, quality, and energy. We explore different algorithms based on graph search, auctions, and distributed optimization for decision-making and action evaluation. We test these in a simulated surveillance scenario with unforeseen emergency trajectories.
We introduce a computational version of the generalized quasi-variational inequality problem and study its computational complexity, in particular proving that it is PPAD-complete. We also consider applications to multi-leader-follower games, a domain traditionally marked by the absence of general solutions. However, through the use of relaxation techniques, we obtain versions of these problems which may be formulated in terms of quasi-variational inequalities, allowing us to obtain PPAD-completeness for such games.
Banks publish daily a list of available securities/assets (axe list) to selected clients to help them effectively locate Long (buy) or Short (sell) trades at reduced financing rates. This reduces costs for the bank, as the list aggregates the bank's internal firm inventory per asset for all clients of long as well as short trades. However, this is somewhat problematic: (1) the bank's inventory is revealed; (2) trades of clients who contribute to the aggregated list, particularly those deemed large, are revealed to other clients. Clients conducting sizable trades with the bank and possessing a portion of the aggregated asset exceeding 50% are considered to be concentrated clients. This could potentially reveal a trading concentrated client's activity to their competitors, thus providing an unfair advantage over the market. Atlas-X Axe Obfuscation, powered by new differential private methods, enables a bank to obfuscate its published axe list on a daily basis while under continual observation, thus maintaining an acceptable inventory Profit and Loss (P&L) cost pertaining to the noisy obfuscated axe list while reducing the clients' trading activity leakage. Our main differential private innovation is a differential private aggregator for streams (time series data) of both positive and negative integers under continual observation. For the last two years, the Atlas-X system has been live in production across three major regions-USA, Europe, and Asia-at J.P. Morgan, a major financial institution. To our knowledge, it is the first differential privacy solution to be deployed in the financial sector. We also report benchmarks of our algorithm based on (anonymous) real and synthetic data to showcase the quality of our obfuscation and its success in production.
A society graph, as considered by [Faliszewski et al., IJCAI 2018], is a graph corresponding to an election instance where every possible ranking is a node, and the weight of such a node is given by the number of voters whose vote correspond to the said ranking. A natural diffusion process on this graph is defined, and an immediate question that emerges is whether there is a diffusion path that leads to a particular candidate winning according to a certain voting rule-this turns out to be NP-complete. In this contribution, we consider the setting when votes are approval ballots, as opposed to rankings-and we consider both the possible and necessary winner problems. We demonstrate that it is possible to efficiently determine if a candidate is a possible winner (i.e, if there exists a diffusion path along which a given candidate wins the election) if the underlying society graph is a star (i.e, tree of diameter at most two), while the problem is NP-complete for trees of diameter d for d > 2. Analogously, we show that it is possible to efficiently determine if a candidate is a necessary winner (i.e, a winner for every possible diffusion path) if the underlying society graph is a star, while the problem is coNP-complete for trees of diameter d for d > 2. We also show the following results on structured graphs for the possible winner problem: the problem is strongly NP-complete on a disjoint union of paths, and on trees of constant diameter. We also report preliminary experiments from an ILP-based implementation.
