Coordination Of Multiple Behaviors Acquired By A Vision-Based Reinforcement Learning

A method is proposed which accomplishes a w- hole task consisting of plural subtasks by coordinat- ing multiple behaviors acquired by a vision-based rein- forcement learning. First, individual behaviors which achieve the corresponding subtasks are independently acquired by Q-learning, a widely used reinforcement learning method. Each learned behavior can be repre- sented by an action-value function in terms of state of the environment and robot action. Next, three kinds of coordinations of multiple behaviors are considered; simple summation of dierent action-value function- s, switching action-value functions according to situ- ations, and learning with previously obtained action- value functions as initial values of a new action-value function. A task of shooting a ball into the goal avoid- ing collisions with an enemy is examined. The task can be decomposed into a ball shooting subtask and a collision avoiding subtask. These subtasks should be accomplished simultaneously, but they are not inde- pendent of each other. Three kinds of coordinations are compared with each other by computer simulations and our on-going real experiments are explained.