Habituation, a non-associative learning widely observed across phylogeny, is fundamental for adaptation and, thus, survival of living organisms. This paper investigates the main characteristics of habituation in order to present three new computational models inspired by habituation. We develop these models as part of the Iterant Deformable Sensorimotor Medium (IDSM), a recently developed abstract model of behavior formation. The characteristics of these models are studied and analyzed. Our long-term objective is to research new unsupervised learning mechanisms for artificial learning agents.

Sensitization, a non-associative learning widely observed across phylogeny, is fundamental for adaptation and, thus, survival of living organisms. This paper investigates one form of sensitization, namely pseudo-conditioning, in order to present a new computational model inspired by its characteristics. We develop this model as part of the Iterant Deformable Sensorimotor Medium (IDSM), a recently developed abstract model of sensorimotor behavior formation. The characteristics of the presented model are studied and analyzed in the light of our long term objective of investigating new unsupervised learning mechanisms for artificial autonomous agents.

In this paper, we propose a contribution in the field of Reinforcement Learning (RL) with continuous state space.Our work is along the line of previous works involving a vector quantization algorithm for learning the state space representation on top of which a function approximation takes place.In particular, our contribution compares the performances of the Kohonen SOM and the Rougier DSOM with the Göppert function approximation scheme on both the mountain car problem.We give a particular focus to DSOM as it is less sensitive to the density of inputs and opens interesting perspectives in RL.

In the framework of model-free deep reinforcement learning with continuous sensorimotor space, we propose a new type of transfer learning, inspired by the child development, where the sensorimotor space of an agent grows while it is learning a policy. To decide how the dimensions grow in our neural network based actor-critic, we add new developmental layers to the neural networks which progressively uncover some dimensions of the sensorimotor space following an Intrinsic Motivation heuristic. To mitigate the catastrophic forgetting problem, we take inspiration from the Elastic Weight Constraint to regulate the learning of the neural controller. We validate our approach using two state-of-the-art algorithms (DDPG and NFAC) on two high-dimensional environment benchmarks (Half-Cheetah and Humanoid). We show that searching first for a suboptimal solution in a subset of the parameter space, and then in the full space, is helpful to bootstrap learning algorithms, and thus reach better performances in fewer episodes.

In the framework of model-free deep reinforcement learning with continuous sensorimotor space, we propose a new type of transfer learning, inspired by the child development, where the sensorimotor space of an agent grows while it is learning a policy. To decide how the dimensions grow in our neural network based actor-critic, we add new developmental layers to the neural networks which progressively uncover some dimensions of the sensorimotor space following an Intrinsic Motivation heuristic. To mitigate the catastrophic forgetting problem, we take inspiration from the Elastic Weight Constraint to regulate the learning of the neural controller. We validate our approach using two state-of-the-art algorithms (DDPG and NFAC) on two high-dimensional environment benchmarks (Half-Cheetah and Humanoid). We show that searching first for a suboptimal solution in a subset of the parameter space, and then in the full space, is helpful to bootstrap learning algorithms, and thus reach better performances in fewer episodes.

Quand les ´etats d’un processus ne sont pas Markoviens (POMDP par exemple), la convergence des algorithmes d’apprentissage par renforcement n’est pas garantie. Une solution est de reconstruire un processus Markovien en partant de la s´equence des ´etats. Dans ce but, nous explorons les capacit´es d’architectures r´ecurrentes qui s’appuient sur des cartes neuronales autoorganisatrices pour apprendre `a pr´edire des s´equences d’observations issues de HMM.

A novel reinforcement learning algorithm that deals with both continuous state and action spaces is proposed. Domain knowledge requirements are kept minimal by using non-linear estimators and since the algorithm does not need prior trajectories or known goal states. The new actor-critic algorithm is on-policy, offline and model-free. It considers discrete time, stationary policies, and maximizes the discounted sum of rewards. Experimental results on two common environments, showing the good performance of the proposed algorithm, are presented.

A new off-policy, offline, model-free, actor-critic reinforcement learning algorithm dealing with continuous environments in both states and actions is presented. It addresses discrete time problems where the goal is to maximize the discounted sum of rewards using stationary policies. Our algorithm allows to trade-off between data-efficiency and scalability. The amount of a priori knowledge is kept low by: (1) using neural networks to learn both the critic and the actor, (2) not relying on initial trajectories provided by an expert, and (3) not depending on known goal states. Experimental results compare data-efficiency to 4 state-of-the-art algorithms on three benchmark environments. This article largely reproduces a previous work [34] by adding a higher dimensional environment, improving control architectures and provides batch normalization for others state-of-the-art algorithms.

We propose an architecture for self-motivated agents allowing them to construct their own knowledge of objects and of geometrical properties of space through interaction with their environment. Self-motivation is defined here as a tendency to experiment and to respond to behavioral opportunities afforded by the environment. Interactions have predefined valences that specify inborn behavioral preferences. The long-term goal is to design agents that construct their own knowledge of their environment through experience, rather than exploiting pre-coded knowledge. Over time, the agent learns relations between elements of the environment that afford its interactions, and its perception of these elements, in the form of data structures called signatures of interactions. These signatures allow the agent to attribute a low level semantics to elements that constitute its environment based on valences of interactions, without predefined knowledge about these elements and regardless of the number of element types. Signatures of interaction are then used to localize elements in space and to construct data structures that characterize spatial properties of space, called signatures of places and signatures of presence. Signatures of place and of presence characterize space using interactions rather than geometrical or topological properties. The agent uses these structures to maintain an egocentric representation of affordances of the surrounding environment, without any preconception about the elements that compose the environment, and without using notions of geometrical space. Experiments with simulated agents show that they learn to behave in their environment, taking into account multiple surrounding objects, reaching or avoiding objects according to the valence of the interactions that they afford.

A new off-policy, offline, model-free, actor-critic reinforcement learning algorithm dealing with continuous environments in both states and actions is presented. It addresses discrete time problems where the goal is to maximize the discounted sum of rewards using stationary policies. Our algorithm allows to trade-off between data-efficiency and scalability. The amount of a priori knowledge is kept low by: (1) using neural networks to learn both the critic and the actor, (2) not relying on initial trajectories provided by an expert, and (3) not depending on known goal states. Experimental results show better data-efficiency than 4 state-of-the-art algorithms on two benchmark environments.

Tracking and understanding moving pedestrian behaviors is of major concern for a growing number of applications. Classical approaches either consider both problems separately or treat them simultaneously on the basis of limited contextual graphical models. In this paper, we consider tackling both problems jointly based on richer contextual information issued from agent-based behavioral simulators designed for realistically reproducing human behaviors within complex environments. We focus on the single target case and experimentally show that the proposed approach keeps good performances even in case of long periods of occlusion.

In this paper, we describe and evaluate an original Monte Carlo JPDAF for tracking interacting autonomous targets in a cluttered environment. The originality of the proposed algorithm consists in reducing the complexity of the prediction step by selecting and separately updating groups of targets in interaction. The complexity of the correction step is addressed by Data Association and a gating procedure as found in literature. The main assumptions we make in this paper are (i) that the evolution of the state of each target only depends on the states of all the targets at the previous time step and (ii) that a generic simulator or a function modeling the targets' behaviors and their mutual interactions is available. We also build an approximate interaction graph between targets on the fly on the basis of simple information like their location, as it has been done in previous work. Experiments show that representing interactions this way can lead to good tracking efficiency with low computational cost.

Tracking and understanding moving pedestrian behaviors is of major concern for a growing number of applications. Classical approaches either consider the two problems separately or treat them simultaneously while relying on limited context-based graphical models. In this paper, we present an approach to tackle both the problems conjointly based on richer contextual information issued from agent-based behavioral simulators which aim to realistically reproduce human behaviors within complex environments. We focus on the special case of a single target and experimentally show that the proposed approach manages to track a single pedestrian with complex behavior even in case of long periods of occlusion.

Tracking and understanding moving pedestrian behaviors is of major concern for a growing number of applications. This problem, known as difficult, is more complex when the considered environment is not fully under sensory coverage. Classical approaches either focus on location estimation or attempt to build the relationship between possible activities in the environment and reason on it, which may turn out to be inadequate. In this paper, we propose an approach based on behavioral models from the situated artificial intelligence field, which aim to realistically reproduce human behaviors within complex environments. We focus on the special case of a single target and experimentally show that we are performing well even in case of long periods of occlusion.