Asymmetric Self-Play for Learning Robust Human-Robot Interaction on Crowd Navigation Tasks

A robot can make a sound to aware nearby pedestrians during its navigation, which often results in a more efficient and safer trajectory in a crowded environment. However, it is challenging to integrate such interaction capability into an existing robot navigation policy. In this paper, we propose a deep reinforcement learning (DRL) approach for integration, which results in effective interactive robot navigation policies in crowded environments. In particular, we specify the interaction capability by a set of high-level actions with flexible control. Then the navigation policy can be considered as a specific implementation of the ‘move’ action. Based on these high-level actions, we can train a robust human-robot interaction policy via asymmetric self-play, where the robot and some pedestrians, considered as naughty kids, play a game. Then the interaction policy can not only interact with pedestrians who cooperate with the robot but also with pedestrians who try to block the robot. We evaluate our approach in various simulation environments and compare it with multiple approaches. The experimental results show that our approach is robust and performs well in dense environments that are challenging for others. We also deploy the trained policy on a robot and evaluate its performance in multiple real-world crowded environments. A demonstration video is available online at https://youtu.be/x32YmivsIh0.