Learning Hierarchical Skills for Game Agents from Video of Human Behavior

Developing autonomous agents for computer games is often a lengthy and expensive undertak- ing that requires manual encoding of detailed and complex knowledge. In this paper we show how to acquire hierarchical skills for controlling a team of simulated football players by observing video of college football play. We then demonstrate the re- sults in the Rush 2008 football simulator, showing that the learned skills have highfidelity with respect to the observed video and are robust to changes in the environment. Finally, we conclude with discus- sions of this work and of possible improvements. skills are acquired in a cumulative manner, with new skills building on those previously learned, and are based on con- ceptual knowledge that includes temporal constraints. The learned skills also reproduce the observed behavior faithfully with comparable efficacy in spite of differences between the observation and demonstration environments. We begin by introducing the Rush 2008 football simulator, which we use to demonstrate our approach. We then briefly review the ICARUS agent architecture, and present our work on acquiring structured domain knowledge from observed hu- man behavior within this framework. Next, we outline the video preprocessing steps, the application of ICARUS to the processed video, and the application of the acquired skills in Rush. Finally, we conclude with a summary of related work and directions for future development.