The introduction of subtasks holds the promise of promoting coordination in scenarios without communication. Instead of manually defined subtasks, recent studies attempt to decompose the overall task and allocate subtasks to agents automatically, but it remains unclear how to acquire a set of proficient subtask representations. In essence, the subtasks serve as auxiliary signals that assist agents in deducing the broader context from limited observations. To embed maximal information into subtask representations, we propose to first learn a vector quantization variational autoencoder which takes individual observations of agents as inputs and reconstructs the global state based on their assigned subtasks as latent variables. Next, the informative representations can be readily integrated into various classic multi-agent reinforcement learning frameworks to facilitate insightful decisions of agents. Experiments on StarCraft II micro-war challenges and Google Research Football have demonstrated that our method learns reasonable and informative subtask representations, which facilitate the decision-making of agents and significantly improve the overall performance.
分享
引用
