PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations.
2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(2022)
Abstract
Evolution Strategy (ES) algorithms have shown promising results in training complex robotic control policies due to their massive parallelism capability, simple implementation, effective parameter-space exploration, and fast training time. However, a key limitation of ES is its scalability to large capacity models, including modern neural network architectures. In this work, we develop Predictive Information Augmented Random Search (PI-ARS) to mitigate this limitation by leveraging recent advancements in representation learning to reduce the parameter search space for ES. Namely, PI-ARS combines a gradient-based representation learning technique, Predictive Information (PI), with a gradient-free ES algorithm, Augmented Random Search (ARS), to train policies that can process complex robot sensory inputs and handle highly nonlinear robot dynamics. We evaluate PI-ARS on a set of challenging visual-locomotion tasks where a quadruped robot needs to walk on uneven stepping stones, quincuncial piles, and moving platforms, as well as to complete an indoor navigation task. Across all tasks, PI-ARS demonstrates significantly better learning efficiency and performance compared to the ARS baseline. We further validate our algorithm by demonstrating that the learned policies can successfully transfer to a real quadruped robot, for example, achieving a 100% success rate on the real-world stepping stone environment, dramatically improving prior results achieving 40% success.
MoreTranslated text
Key words
ARS baseline,complex robot sensory inputs,complex robotic control policies,effective parameter-space exploration,evolution strategy algorithms,evolution-learned visual-locomotion,gradient-based representation,gradient-free ES algorithm,learned policies,nonlinear robot dynamics,parameter search space,PI-ARS,predictive information augmented random search,predictive information representations,quadruped robot,representation learning,visual-locomotion tasks
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined