IEEE International Conference on Development and Learning : [proceedings]

Robots are getting increasingly more present in many spheres of human life, making the need for robots that can successfully engage in natural social interactions with humans paramount. Successful human-robot interaction could be achieved more effectively if robots could act predictably and could predict the humans' actions. If robots could represent human partners and generate behaviors that are in line with the partners' expectations based on human's mental models of interdependent action, human agents would be able to apply predictive and adaptive mechanisms acquired in human interactions to interact with robots effectively. How could robots be predictable and be capable of predicting human behavior? We propose that this could be achieved by having an internal representation of both oneself and the other agent, that is by equipping the robot with the ability to co-represent. Here, co-representation refers to the representation of the partner's actions alongside one's own actions. Although co-representation constitutes an essential process for successful human social interaction, as it supports understanding of others' actions, to date co-representation processes have only scarcely been integrated into robotic platforms. We highlight the state-of-the-art findings on co-representation in social robotics, discuss current research limitations and open issues for creating computational models of co-representation in robots, and put forward the idea that predictive learning might constitute a particularly promising framework to build models of co-representing robots. Overall, in this article, we offer an integrated view of the state-of-the-art findings in robotics literature on co-representation and outline directions for future research, with the aim to boost success in building robots equipped with co-representation models fit for smooth social interactions.

How do cognitive agents decide what is the relevant information to learn and how goals are selected to gain this knowledge? Cognitive agents need to be motivated to perform any action. We discuss that emotions arise when differences between expected and actual rates of progress towards a goal are experienced. Therefore, the tracking of prediction error dynamics has a tight relationship with emotions. Here, we suggest that the tracking of prediction error dynamics allows an artificial agent to be intrinsically motivated to seek new experiences but constrained to those that generate reducible prediction error.We present an intrinsic motivation architecture that generates behaviors towards self-generated and dynamic goals and that regulates goal selection and the balance between exploitation and exploration through multi-level monitoring of prediction error dynamics. This new architecture modulates exploration noise and leverages computational resources according to the dynamics of the overall performance of the learning system. Additionally, it establishes a possible solution to the temporal dynamics of goal selection. The results of the experiments presented here suggest that this architecture outperforms intrinsic motivation approaches where exploratory noise and goals are fixed and a greedy strategy is applied.

How the sensorimotor experience of an agent can be organized into abstract symbol-like structures to enable effective planning and control is an open question. In the literature, there are many studies that start by assuming the existence of some symbols and ‘ground’ those onto continuous sensorimotor signals. There are also works that aim to facilitate the emergence of symbol-like representations by using specially designed machine learning architectures. In this paper, we investigate whether a deep reinforcement learning system that learns a dynamic task would facilitate the formation of high-level neural representations that might be considered as precursors of symbolic representation, which could be exploited by higher level neural circuits for better control and planning. The results indicate that without even explicit design to promote such representations, neural responses emerge that may serve as the basis of abstract symbol-like representations.

In robotics, methods and softwares usually require optimizations of hyperparameters in order to be efficient for specific tasks, for instance industrial bin-picking from homogeneous heaps of different objects. We present a developmental framework based on long-term memory and reasoning modules (Bayesian Optimisation, visual similarity and parameters bounds reduction) allowing a robot to use meta-learning mechanism increasing the efficiency of such continuous and constrained parameters optimizations. The new optimization, viewed as a learning for the robot, can take advantage of past experiences (stored in the episodic and procedural memories) to shrink the search space by using reduced parameters bounds computed from the best optimizations realized by the robot with similar tasks of the new one (e.g. bin-picking from an homogenous heap of a similar object, based on visual similarity of objects stored in the semantic memory). As example, we have confronted the system to the constrained optimizations of 9 continuous hyperparameters for a professional software (Kamido) in industrial robotic arm bin-picking tasks, a step that is needed each time to handle correctly new object. We used a simulator to create bin-picking tasks for 8 different objects (7 in simulation and one with real setup, without and with meta-learning with experiences coming from other similar objects) achieving goods results despite a very small optimization budget, with a better performance reached when meta-learning is used (84.3 % vs 78.9 % of success overall, with a small budget of 30 iterations for each optimization) for every object tested (p-value=0.036).

Robots are becoming more and more present in our daily life operating in complex and unstructured environments. To operate autonomously they must adapt to continuous scene changes and therefore must rely on an incessant learning process. Deep learning methods have reached state-of-the-art results in several domains like computer vision and natural language processing. The success of these deep networks relies on large representative datasets used for training and testing. But one limitation of this approach is the sensitivity of these networks to the dataset they were trained on. These networks perform well as long as the training set is a realistic representation of the contextual scenario. For robotic applications, it is difficult to represent in one dataset all the different environments the robot will encounter. On the other hand, a robot has the advantage to act and to perceive in the complex environment. As a consequence when interacting with humans it can acquire a substantial amount of relevant data, that can be used to perform learning. The challenge we addressed in this work is to propose a computational architecture that allows a robot to learn autonomously from its sensors when learning is supported by an interactive human. We took inspiration on the early development of humans and test our framework on the task of localisation and recognition of objects. We evaluated our framework with the humanoid robot iCub in the experimental context of a realistic interactive scenario. The human subject naturally interacted with the robot showing objects to the iCub without supervision in the labelling. We demonstrated that our architecture can be used to successfully perform transfer learning for an object localisation network with limited human supervision and can be considered a possible enhancement of traditional learning methods for robotics.

Robots that cohabitate in social spaces must abide by the same behavioural cues humans follow, including interpersonal distancing. Proxemics investigates the appropriate distances and the impact of factors affecting it, such as gender and age. This paper investigates people's attitudes towards a robot that can learn Proxemics rules by gauging direct individual feedback from a person, and utilizing it in a reinforcement learning framework. Previous learning attempts have relied on larger robots, for which physical safety is a primary concern. In contrast, our study uses a handheld sized robot that allows us to focus on the impact of distance on engageability in dialogue. General consensus between interviewees was a feeling of ease and safety during interactions, as well as disparity regarding the invasion of personal space, which was influenced by cultural background.

We present an analysis of how children between 4-and 9-years-old give directions to a robot. Thirty-eight children in this age range participated in a direction giving game with a virtual robot and with their caregiver. We considered two different viewpoints (aerial and in-person) and three different affordances (non-humanoid robot, caregiver with eyes closed, and caregiver with eyes open). We report on the frequency of commands that children used, the complexity of the commands, and the navigation styles children used at different ages. We found that pointing and gesturing decreased with age, while “left-right” directions and the use of distances increased with age. From this, we make several recommendations for robot design that would enable a robot to successfully follow directions from children of different ages, and help advance children's direction giving.

The concept of time perception is used to describe the phenomenological experience of time. There is strong evidence that dopaminergic neurons are involved in the timing mechanisms responsible for time perception. The phasic activity of these neurons resembles the behavior of the reward prediction error in temporal-difference learning models. Therefore, these models are used to replicate the neuronal behaviour of the dopamine system and corresponding timing mechanisms. However, time perception has also been shown to be shaped by time estimation mechanisms from external stimuli. In this paper we propose a framework that combines these two principles, in order to provide temporal cognition abilities to intelligent systems such as robots. A time estimator based on observed environmental stimuli is combined with a reinforcement learning approach, using a feature representation called Microstimuli to replicate dopaminergic behaviour. The elapsed time perceived by the robot is estimated by modeling sensor measurements as Gaussian processes to capture the second-order statistics of the natural environment. The proposed framework is evaluated on a simulated robot that performs a temporal discrimination task originally performed by mice. The ability of the robot to replicate the timing mechanisms of the mice is demonstrated by the fact that both exhibit the same ability to classify the duration of intervals.

Automating robot behavior in a specific situation is an active area of research. There are several approaches available in the literature of robotics to cater for the automatic behavior of a robot. However, when it comes to humanoids or human-robot interaction in general, the area has been less explored. In this paper, a pseudo-randomization approach has been introduced to automatize the gestures and facial expressions of an interactive humanoid robot called ROBIN based on its mental state. A significant number of gestures and facial expressions have been implemented to allow the robot more options to perform a relevant action or reaction based on visual stimuli. There is a display of noticeable differences in the behaviour of the robot for the same stimuli perceived from an interaction partner. This slight autonomous behavioural change in the robot clearly shows a notion of automation in behaviour. The results from experimental scenarios and human-centered evaluation of the system help validate the approach.

Humans manage to communicate action intentions in a non-verbal way, through body posture and movement. We start from this observation to investigate how a robot can decode a human's non-verbal cues during the manipulation of an object, with specific physical properties, to learn the adequate level of "carefulness" to use when handling that object. We construct dynamical models of the human behaviour using a human-to-human handover dataset consisting of 3 different cups with different levels of fillings. We then included these models into the design of an online classifier that identifies the type of action, based on the human wrist movement. We close the loop from action understanding to robot action execution with an adaptive and robust controller based on the learned classifier, and evaluate the entire pipeline on a collaborative task with a 7-DOF manipulator. Our results show that it is possible to correctly understand the "carefulness" behaviour of humans during object manipulation, even in the pick and place scenario, that was not part of the training set.

We present a model of the self-calibration of active binocular vision comprising the simultaneous learning of visual representations, vergence, and pursuit eye movements. The model follows the principle of Active Efficient Coding (AEC), a recent extension of the classic Efficient Coding Hypothesis to active perception. In contrast to previous AEC models, the present model uses deep autoencoders to learn sensory representations. We also propose a new formulation of the intrinsic motivation signal that guides the learning of behavior. We demonstrate the performance of the model in simulations.

Individuals with cerebral palsy use electric wheelchairs due to their abnormal gait caused by paralysis and other symptoms. However, it is difficult for them to operate the wheelchair joystick because of their suddenly occurring uncontrollable, involuntary movements and the difficulty they have maintaining their posture. In this study, we developed a control system, which suppresses the effects of involuntary movement. This system is capable of controlling electric wheelchairs as intended by individuals with tension-athetosis-type cerebral palsy. We demonstrated the experiments to compare the stability of operation by normal system and the proposed system. Finally, we showed the effectiveness of the proposed system in the straight running experiment.

Robotic manipulation has been generally applied to particular settings and a limited number of known objects. In order to manipulate novel objects, robots need to be capable of discovering the physical properties of objects, such as the center of mass, and reorienting objects to the desired pose required for subsequent actions. In this work, we proposed a computationally efficient 2-stage framework for planar pushing, allowing a robot to push novel objects to a specified pose with a small amount of pushing steps. We developed three modules: Coarse Action Predictor (CAP), Forward Dynamic Estimator (FDE), and Physical Property Estimator (PPE). The CAP module predicts a mixture of Gaussian distribution of actions. FDE learns the causality between action and successive object state. PPE based on Recurrent Neural Network predicts the physical center of mass (PCOM) from the robot-object interaction. Our preliminary experiments show promising results to meet the practical application requirements of manipulating novel objects.

Applying intelligence to a group of simple robots known as swarm robots has become an exciting technology in assisting or replacing humans to fulfil complex, dangerous and harsh missions. However, building a strategy for a swarm to thrive in a dynamic environment is challenging because of control decentralisation and interactions between agents. The decision-making process in a robotic task commonly takes place in sequential stages. By understanding the subsequent action-reaction process, a strategy to make optimal decisions in a respective environment can be learnt. Hence, using the concept of epigenetic inheritance, novel evolutionary-learning mechanisms for a swarm will be discussed in this paper. Reinforcement evolutionary learning using epigenetic inheritance (RELEpi) is proposed in this article. This method utilizes reward, temporal difference and epigenetic inheritance to approximate optimal action and behaviour policies. The proposed method opens possibilities to combine reward-based learning and evolutionary methods as a stacked process where histone value is used rather than fitness function. The formulation consists of methylation and epigenetic mechanisms, inspired by the epigenome studies. The methylation process helps the accumulation of the reward to histone value of the gene. Epigenetic mechanisms give the ability to mate genetic information along with their histone value.

Traditional methods for stereo-disparity detection use explicit search between the left and right images. Although such methods are simple and intuitive for understanding, they suffer from degeneracies when the search window contains weak texture. Developmental Networks (DNs) are task-nonspecific and modality-nonspecific learning engines. Because they are general-purpose learners, they have a potential to deal with many types of degeneracies in intelligent systems. This work presents two novel mechanisms to deal with degeneracies: volume dimension and subwindow voting. While developmental stereo-disparity detection has been tested on simulated stereo images in our prior publications, it has never been tested on the real world. This paper reports our system, 3DEye, which is the first to have filled this void. The algorithm, software, graphical user interface, training, performance, and update rates on CPU and GPU, respectively, on a Sony G8142 mobile phone are reported. Many deep learning methods that use error back-propagation suffer from the controversy of “post-selection” using the test set [1], to select one from many networks to report. In contrast, all randomly initialized DNs are performance-equivalent, no “post-selection” using test set. Possible future improvements for practical real-world and real-time applications are discussed.

Both biological and artificial agents need to coordinate their behavior to suit various needs at the same time. Reconciling conflicts of different needs and contradictory interests such as self-preservation and curiosity is the central difficulty arising in the design and modelling of need and value systems. Current models of multi-objective reinforcement learning do either not provide satisfactory power to describe such conflicts, or lack the power to actually resolve them. This paper aims to promote a clear understanding of these limitations, and to overcome them with a theory-driven approach rather than ad hoc solutions. The first contribution of this paper is the development of an example that demonstrates previous approaches' limitations concisely. The second contribution is a new, non-linear objective function design, MORE, that addresses these and leads to a practical algorithm. Experiments show that standard RL methods fail to grasp the nature of the problem and ad-hoc solutions struggle to describe consistent preferences. MORE consistently learns a highly satisfactory solution that balances contradictory needs based on a consistent notion of optimality.

We tested whether the observation of motor action encoding social motor intention would cause the spontaneous processing of a complementary response when performed by a humanoid robot. We designed the robot's arm and upper body movements to manifest the kinematic profiles of human individual and social motor intention and designed a simple task that involved robot and human placing blocks on a table sequentially. Our results show that the behavior of the human can be modulated by human kinematics as encoded in a robot's movement. In several cases human subjects reciprocated movement that displayed social motor intention with movements showing a similar kinematic profile while attempting to make eye contact and engaging in turn-taking behaviour during the task. This suggests a novel approach in the design of HRI based in motor processing that promises to be ecologically valid, cheap, automatic, fast, resilient, intuitive, and computationally simple.

The advancement of technology has brought many benefits to robotics. Today, it is possible to have robots equipped with many sensors that collect different kinds of information on the environment all time. However, this brings a disadvantage: the increase of information that is received and needs to be processed. This computation is too expensive for robots and is very difficult when it has to be performed online and involves a learning process. Attention is a mechanism that can help us address the most critical data at every moment and is fundamental to improve learning. This paper discusses the importance of attention in the learning process by evaluating the possibility of learning over the attentional space. For this purpose, we modeled in a cognitive architecture the essential cognitive functions necessary to learn and used bottom-up attention as input to a reinforcement learning algorithm. The results show that the robot can learn on attentional and sensorial spaces. By comparing various action schemes, we find the set of actions for successful learning.

Universal Turing Machines are well known in computer science but they are about manual programming for general purposes. Although human children perform conscious learning (learning while being conscious) from infancy, it is unknown that Universal Turing Machines can facilitate not only our understanding of Autonomous Programming For General Purposes (APFGP) by machines, but also enable early-age conscious learning. This work reports a new kind of AI-conscious learning AI from a machine's “baby” time. Instead of arguing what static tasks a conscious machine should be able to do during its “adulthood”, this work suggests that APFGP is a computationally clearer and necessary criterion for us to judge whether a machine is capable of conscious learning so that it can autonomously acquire skills along its “career path”. The results here report new concepts and experimental studies for early vision, audition, natural language understanding, and emotion, with conscious learning capabilities that are absent from traditional AI systems.

This paper describes a unified computational model for the joint development of early visual representations and the control of three types of eye movements, i.e., vergence, cyclovergence, and saccades. The model is based on the Active Efficient Coding (AEC) framework, an extension of Barlow's efficient coding hypothesis to active perception. AEC describes the joint learning of sensory encoding and behavioral control. The present work relaxes the assumptions made in our previous work by learning vergence, cyclovergence, and saccades all from random initialization. Our results also demonstrate the importance of the interaction between the learning of these eye movements in terms of learning speed and accuracy. Overall, we find that AEC provides a parsimonious framework to account for the simultaneous learning of active vision skills.