Reinforcement Learning: an Introduction.

In which we try to give a basic intuitive sense of what reinforcement learning is and how it diiers and relates to other elds, e.g., supervised learning and neural networks, genetic algorithms and artiicial life, control theory. Intuitively, RL is trial and error (variation and selection, search) plus learning (association, memory). We argue that RL is the only eld that seriously addresses the special features of the problem of learning from interaction to achieve long-term goals.] The idea that we learn by interacting with our environment is probably the rst to occur to us when we think about the nature of learning. When an infant plays, waves its arms, or looks about, it has no explicit teacher, but it does have a direct sensorimotor connection to its environment. Exercising this connection produces a wealth of information about cause and eeect, about the consequences of actions, and about what to do in order to achieve goals. This interaction is undoubtedly a major contributor to the infant's developing sense of its environment and of its own role in it. Experience remains a powerful teacher as the infant grows into a child and an adult, although the nature interaction changes signiicantly over time. Whether we are learning to drive a car or to hold a conversation, we are all acutely aware of how our environment responds to what we do, and we seek to innuence its behavior. Learning from interaction is a foundational idea underlying nearly all theories of learning. Reinforcement learning a computational approach to the study of learning from interaction. The last decade has seen the study of reinforcement learning develop into an unusually multidisciplinary eld; it includes researchers specializing in artiicial intelligence, psychology, control engineering, operations research,