Grounding Natural Language Instructions in Industrial Robotics

Most industrial scenarios are characterized by the fact that human and robots operate in completely separated spaces. Industrial manipulators, for example, are usually constrained into cells where humans can enter only for maintenance, and in most cases, the robotic system completely stops its task when the human enters the cell. Moreover, the interaction between industrial robots and operators is very structured and usually only unidirectional. In this work, we aim to close the loop between these heterogeneous agents by implementing a layer of interaction between the robotic system and the human user. In our system, the interaction is achieved by means of spoken natural language, a complex and high-level interface that usually is not adopted in such controlled and structured environments. The introduction of such interface improves the ease of use of the system, giving the opportunity to successfully accomplish complex tasks even when there are no expert users in the loop. In order to achieve the most natural and effective interaction between human and robot, we implemented a language understanding engine explicitly designed and trained for robots, and a knowledge representation architecture based on semantic maps. The proposed system is able to deal with the intrinsic complexity and ambiguity of natural language by explicitly taking into account the problem of disambiguating potentially overlapping actions. The system has been tested over multiple scenarios and different applications of human-robot collaboration, from collaborative environment exploration to collaborative part assembly.