Damage recovery is critical for autonomous robots that need to operate for a long time without assistance. Most current methods are complex and costly because they require anticipating potential damage in order to have a contingency plan ready. As an alternative, we introduce the T-resilience algorithm, a new algorithm that allows robots to quickly and autonomously discover compensatory behavior in unanticipated situations. This algorithm equips the robot with a self-model and discovers new behavior by learning to avoid those that perform differently in the self-model and in reality. Our algorithm thus does not identify the damaged parts but it implicitly searches for efficient behavior that does not use them. We evaluate the T-resilience algorithm on a hexapod robot that needs to adapt to leg removal, broken legs and motor failures; we compare it to stochastic local search, policy gradient and the self-modeling algorithm proposed by Bongard et al. The behavior of the robot is assessed on-board thanks to an RGB-D sensor and a SLAM algorithm. Using only 25 tests on the robot and an overall running time of 20 min, T-resilience consistently leads to substantially better results than the other approaches.

In robotics, gradient-free optimization algorithms (e.g. evolutionary algorithms) are often used only in simulation because they require the evaluation of many candidate solutions. Nevertheless, solutions obtained in simulation often do not work well on the real device. The transferability approach aims at crossing this gap between simulation and reality by making the optimization algorithm aware of the limits of the simulation. In the present paper, we first describe the transferability function, that maps solution descriptors to a score representing how well a simulator matches the reality. We then show that this function can be learned using a regression algorithm and a few experiments with the real devices. Our results are supported by an extensive study of the reality gap for a simple quadruped robot whose control parameters are optimized. In particular, we mapped the whole search space in reality and in simulation to understand the differences between the fitness landscapes.

Damage recovery is critical for autonomous robots that need to operate for a long time without assistance. Most current methods are complex and costly because they require anticipating each potential damage in order to have a contingency plan ready. An alternative line of thought is to let the robot learn on its own the best behavior for the current situation. If the learning process is open enough, then the robot should be able to discover new compensatory behaviors in situations that have not been foreseen by its designers. Classic reinforcement learning algorithms are hard to apply to low-level robotic problems [11], but evolutionary algorithms (EAs) are good candidates to find original solutions because they can optimize in the continuous domain and work on the structure of controllers, for instance by evolving neural networks. When evolving controllers for robots, EAs are reported to require many hundreds of trials on the robot and to last from two to tens of hours (e.g. [5, 12]). These EAs spend most of their running time in evaluating the quality of controllers by testing them on the target robot. Since, contrary to simulation, reality cannot be sped up, their running time can only be improved by finding strategies to evaluate fewer candidate solutions on the robot. By first learning learning a self-model for the robot, then evolving a controller with this simulation, Bongard et al. [1] designed an algorithm for resilience that makes an important step in this direction. Nevertheless, this algorithm has a few important shortcomings. First, actions and models are

The reality gap, which often makes controllers evolved in simulation inefficient once transferred onto the physical robot, remains a critical issue in evolutionary robotics (ER). We hypothesize that this gap highlights a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: the most efficient solutions in simulation often exploit badly modeled phenomena to achieve high fitness values with unrealistic behaviors. This hypothesis leads to the transferability approach, a multiobjective formulation of ER in which two main objectives are optimized via a Pareto-based multiobjective evolutionary algorithm: 1) the fitness; and 2) the transferability, estimated by a simulation-to-reality (STR) disparity measure. To evaluate this second objective, a surrogate model of the exact STR disparity is built during the optimization. This transferability approach has been compared to two reality-based optimization methods, a noise-based approach inspired from Jakobi's minimal simulation methodology and a local search approach. It has been validated on two robotic applications: 1) a navigation task with an e-puck robot; and 2) a walking task with a 8-DOF quadrupedal robot. For both experimental setups, our approach successfully finds efficient and well-transferable controllers only with about ten experiments on the physical robot.

Wheel-legged hybrid robots promise to combine the efficiency of wheeled robots with the versatility of legged robots: they are able to roll on simple terrains, to dynamically adapt their posture and even to walk on uneven grounds. Although different locomotion modes of such robots have been studied, a pivotal question remains: how to automatically adapt the locomotion mode when the environment changes? We here propose that the robot autonomously discovers its locomotion mode using optimization-based learning. To that aim, we introduce a new algorithm that relies on a forward model and a stochastic multi-objective optimization. Three objectives are optimized: (1) the average displacement speed, (2) the expended energy and (3) the transferability score, which reflects how well the behavior of the robot is in agreement with the predictions of the forward model. This transferability function is approximated by conducting 20 experiments of one second on the real robot during the optimization. In the three investigated situations (flat ground, grass-like terrain, tunnel-like environment), our method found efficient controllers for forward locomotion in 1 to 2 minutes: the robot used its wheels on the flat ground, it walked on the grass-like terrain and moved with a lowered body in the tunnellike environment.

Il est difficile de concevoir des controleurs pour des robots devant fonctionner dans des environnements peu maitrises voire inconnus. Dans cette optique, la robotique evolutionniste cherche a elaborer des methodes de conception automatique de controleurs via un processus d'optimisation ``boite noire'' utilisant des algorithmes evolutionnistes. Les valeurs de performance d'un controleur donne sont alors estimees soit directement sur le robot, soit a l'aide d'une simulation, par essence simplificatrice. Partant du constat qu'evaluer sur le robot et dans toutes ses situations d'utilisation est generalement incompatible avec le nombre d'evaluations requis par de tels processus d'optimisation, nous proposons une approche generale combinant un processus d'optimisation multi-objectif dans un simulateur fixe, un modele de substitution et quelques tests sur le robot. Cette approche par transferabilite peut s'appliquer a tout processus d'optimisation mene dans un environnement simplifie (simulation) pour un environnement complet cible (robot) et recherche les controleurs qui maximisent deux objectifs : la performance dans l'environnement simplifie et un objectif de transferabilite qui indique a quel point le comportement dans l'environnement simplifie est proche de celui dans l'environnement complet. Ce deuxieme objectif est estime par un modele de substitution construit en effectuant quelques experiences de transfert sur le robot pendant l'optimisation. L'approche est validee sur trois problemes de robotique evolutionniste : le passage de la simulation a la realite, l'optimisation de controleurs dotes de capacites de generalisation et l'adaptation d'un robot a son environnement.

Wheel-legged hybrid robots are versatile machines that can employ several locomotion modes; however, automatically choosing the right locomotion mode is still an open problem in robotics. We here propose that the robot autonomously discovers its locomotion mode using a multi-objective evolutionary optimization and a fixed internal model. Three objectives are optimized: (1) the displacement speed computed with the internal model, (2) the predicted expended energy and (3) the transferability score, which reflects how well the behavior of the real robot is in agreement with the predictions of the internal model. This transferability function is actively learned by conducting 20 experiments on the real robot during the optimization. We tested this approach with a wheel-legged robot in three situations (flat ground, grass-like terrain, tunnel-like environment): in each case, the evolutionary algorithm found efficient controllers for forward locomotion in 1 to 2 minutes.

In Evolutionary Robotics (ER), controllers are assessed in a single or a few environments. As a consequence, good performances in new different contexts are not guaranteed. While a lot of ER works deal with robustness, i.e. the ability to perform well on new contexts close to the ones used for evaluation, no current approach is able to promote broader generalisation abilities without any assumption on the new contexts. In this paper, we introduce the ProGAb approach, which is based on the standard three data sets methodology of supervised machine learning, and compare it to state-of-the-art ER methods on two simulated robotic tasks: a navigation task in a T-maze and a more complex ball-collecting task in an arena. In both applications, the ProGAb approach: (1) produced controllers with better generalisation abilities than the other methods; (2) needed two to three times fewer evaluations to discover such solutions.

In evolutionary robotics, controllers are often designed in simulation, then transferred onto the real system. Nevertheless, when no accurate model is available, controller transfer from simulation to reality means potential performance loss. It is the reality gap problem. Unmanned aerial vehicles are typical systems where it may arise. Their locomotion dynamics may be hard to model because of a limited knowledge about the underlying physics. Moreover, a batch identification approach is difficult to use due to costly and time consuming experiments. An automatic identification method is then needed that builds a relevant local model of the system concerning a target issue. This paper deals with such an approach that is based on coevolution of models and tests. It aims at improving both modeling and control of a given system with a limited number of manipulations carried out on it. Experiments conducted with a simulated quadrotor helicopter show promising initial results about test learning and control improvement.

Wheel-legged hybrid robots promise to combine the e ciency of wheeled robots with the versatility of legged robots: they are able to roll on simple terrains, to dynamically adapt their posture and even to walk on uneven grounds. Al- though di erent locomotion modes of such robots have been studied, a pivotal question remains: how to automatically adapt the locomotion mode when the environment changes? We here propose that the robot autonomously discov- ers its locomotion mode using optimization-based learning. To that aim, we introduce a new algorithm that relies on a forward model and a stochastic multi-objective optimization. Three objectives are optimized: (1) the average displacement speed, (2) the expended energy and (3) the transferability score, which re ects how well the behavior of the robot is in agreement with the pre- dictions of the forward model. This transferability function is approximated by conducting 20 experiments of one second on the real robot during the op- timization. In the three investigated situations ( at ground, grass-like terrain, tunnel-like environment), our method found e cient controllers for forward locomotion in 1 to 2 minutes: the robot used its wheels on the at ground, it walked on the grass-like terrain and moved with a lowered body in the tunnel- like environment.