Visual Perception Framework for an Intelligent Mobile Robot

Visual perception is a fundamental capability necessary for intelligent mobile robots to interact properly and safely with the humans in the real-world. Recently, the world has seen revolutionary advances in deep learning has led to some incredible breakthroughs in vision technology. However, research integrating diverse visual perception methods into robotic systems is still in its infancy and lacks validation in real-world scenarios. In this paper, we present a visual perception framework for an intelligent mobile robot. Based on the robot operating system middleware, our framework integrates a broad set of advanced algorithms capable of recognising people, objects and human poses, as well as describing observed scenes. In several challenge scenarios of international robotics competitions using two mobile service robots, the performance and acceptability of the proposed framework are evaluated.