Considering the problem of recognizing non-verbal cues in Human–Robot Interaction applications, this paper proposes a novel real-time unsupervised spike timing neural network for recognition and early detection of spatio-temporal human gestures. Two spiking network classifiers one based on Izhikevich neuron model, and the other one based on Integrate-and-Fire-or-Burst neuron model have been implemented in CUDA, and allow the classification to be performed in real-time. To evaluate the performance of this proposal, we test the case of a physical robot observing air-handwritings of human gesture. The proposed approaches run in real-time, thus they are suitable for human–robot applications; they allow real-time early classifying human gestures and actions while they require a very small number of training samples. In comparing to other prominent techniques, our approaches demonstrate superior accuracy and are suitable for early classification of different types of human actions in time-sensitive mobile applications such as robotics.

This paper addresses the problem of task allocation for multi-robot systems that perform tasks with complex, hierarchical representations which contain different types of ordering constraints and multiple paths of execution. We propose a distributed multi-robot control architecture that addresses the above challenges and makes the following contributions: i) it allows for on-line, dynamic allocation of robots to various steps of the task, ii) it ensures that the collaborative robot system will obey all of the task constraints and iii) it allows for opportunistic, flexible task execution given different environmental conditions. This architecture uses a distributed messaging system to allow the robots to communicate. Each robot uses its own state and team member states to keep track of the progress on a given task and identify which subtasks to perform next using an activation spreading mechanism. We demonstrate the proposed architecture on a team of two humanoid robots (a PR2 and a Baxter) performing hierarchical tasks.

Robot tasks for real-world applications typically involve multiple paths of execution, where the same task can be achieved in different ways. This poses challenges with respect to the representation and execution of such tasks, as enumerating all possible execution paths leads to combinatorial increases in the size of the representation. We present a novel robot control architecture that addresses these challenges. The architecture 1) provides an efficient, compact encoding of tasks with multiple paths of execution, 2) uses the same compact representation as the controller that the robot will use to achieve its goals, 3) allows the robot to dynamically decide which execution path to follow using an activation spreading mechanism that relies on environmental conditions, and 4) provides a mechanism for robustness to changes in the environment during the task execution. We validate our architecture using a humanoid PR2 robot, showing that the robot dynamically selects a path of execution based on the current state of the environment, and is robust to environmental changes.

Robot task representation typically involves a sequential set of steps. This methodology does not generalize to complex real-world tasks. Real-world tasks involve multiple paths of execution, where intra-task dependencies allow for different methods to complete the same task. Representing such complex tasks compactly poses a challenge as enumerating all possible paths results in combinatorial growth. As well, complex tasks benefit from the shared work of multiple agents. Tasks representations should provide a mechanism for multiple agents to communicate and self-organize as a team. We propose a control architecture to address these issues. The architecture 1) provides an efficient an compact encoding of complex tasks, 2) enables robots to infer what the other teammates are doing, 3) allows robots to dynamically decide which execution path to follow using an activation spreading mechanism.