Vision-based reinforcement learning for purposive behavior acquisition

This paper presents a method of vision-based rein- forcement learning by which a robot learns to shoot a ball into a goal, and discusses several issues in applying the reinforcement learning method to a real robot with vision sensor. First, a \state-action deviation" prob- lem is found as a form of perceptual aliasing in con- structing the state and action spaces that reect the outputs from physical sensors and actuators, respec- tively. To cope with this, an action set is constructed in such a way that one action consists of a series of the same action primitive which is successively exe- cuted until the current state changes. Next, to speed up the learning time, a mechanism of Learning form Easy Missions (or LEM) which is a similar technique to \shaping" in animal learning is implemented. LEM reduces the learning time from the exponential order in the size of the state space to about the linear order in the size of the state space. The results of computer simulations and real robot experiments are given.