Runtime Verification of the ARIAC Competition - Can a Robot be Agile and Safe at the same time?

ARIAC (Agile Robotics for Industrial Automation Competition) is a robotic competition which aims to advance robotic agility in industry. Participants in this competition are required to implement a robot control system to overcome agility challenges in a simulated environment. ARIAC comes with a set of score metrics to evaluate the performance of each control system during task execution. In this paper we show how such task-oriented evaluation can be problematic and how the addition of runtime monitors to verify properties given in ISO/TS safety standards can help in reducing the resulting reality gap.