Toward Data Driven Development in RoboCup

Conducting games in RoboCup incurs high cost in terms of effort, time, and money. The scientific outcome, however, is quite limited and often not very conclusive. Especially, analyzing and drawing conclusions about the performance of complex processes like decision making of an individual robot or the behavior on the team level poses a considerable challenge. Collecting more data during the competition games will help to analyze the performance of algorithms, identify errors and areas for improvement, and make more significant statements regarding the performance of the robots. In this work we investigate the possibilities for collection of the large scale RoboCup data and its analysis. We present a system for automatic recording of synchronized videos of RoboCup games and an application for exploration and annotation of large sets of RoboCup-related data. We also present data sets collected during the competitions in 2018 and an algorithm for visual detection and tracking of robots in the RoboCup videos. A first empirical evaluation shows promising results and demonstrates how such data can be integrated and used to validate robot’s behavior.