In this paper, we present a robust system of self-directed autonomous robots evolving in a complex and public spaces and interacting with people. This system integrates high-level skills of environment modeling using knowledge-based modeling and reasoning and scene understanding with robust image and video analysis, distributed autonomous decision-making using Markov decision process and Petri-Net planning, short-term interacting with humans and robust and safe navigation in overcrowding spaces. This system has been deployed in a variety of public environments such as a shopping mall, a center of congress and in a lab to assist people and visitors. The results are very satisfying showing the effectiveness of the system and going beyond just a simple proof of concepts.

We introduce a human-robot interaction framework for robots helping/guiding customers in a shopping mall environment. For that, we design and develop controlled natural languages for customers’ and robots’ questions and instructions. We construct knowledge bases representing general/specific static/dynamic knowledge about shopping malls, to be used in conjunction with the CNLs. We show an application of our framework with a humanoid robot.