This paper presents a novel design of face tracking algorithm and visual state estimation for a mobile robot face tracking interaction control system. The advantage of this design is that it can track a user's face under several external uncertainties and estimate the system state without the knowledge about target's 3D motion-model information. This feature is helpful for the development of a real-time visual tracking control system. In order to overcome the change in skin color due to light variation, a real-time face tracking algorithm is proposed based on an adaptive skin color search method. Moreover, in order to increase the robustness against colored observation noise, a new visual state estimator is designed by combining a Kalman filter with an echo state network-based self-tuning algorithm. The performance of this estimator design has been evaluated using computer simulation. Several experiments on a mobile robot validate the proposed control system.

Elderly and disabled people can experience considerable difficulties when driving a powered wheelchair, especially if they do not possess the fine steering capacities that are required to perform certain manoeuvres, like avoiding obstacles or docking at tables. In order to help these people, several ''intelligent'' wheelchairs have been developed in the past, meaning that a powered wheelchair was endowed with the abilities to provide navigational assistance to its user. In such a scenario, control over the wheelchair is shared between the user and the intelligent assistance. The problem of the existing intelligent wheelchairs is, however, that the rules for assistance are hard-coded and by consequence not adaptable to the personal handicap and needs of a specific user. Therefore, this paper proposes a new, more user-centered approach to shared wheelchair control. The presented framework executes two tasks: it continuously estimates the user's intention and it determines whether the user needs assistance to achieve that intention. An implicit user model is introduced and incorporated in the framework, in order to make the execution of both tasks adaptable to a specific user. This paper presents the proposed framework, along with experimental results in simulation and on a real wheelchair.

Reservoir computing is a new paradigm for using recurrent neural network with a much simpler training method. The key idea is to use a large but fixed recurrent part as a reservoir of dynamic features and to train only the output layer to extract the desired information. We propose to study how pruning some connections from the reservoir to the output layer can help on the one hand to increase the generalization ability, in much the same way as regularization techniques do, and on the other hand to improve the implementability of reservoirs in hardware.

This paper presents a novel design of a robust visual tracking control system, which consists of a visual tracking controller and a visual state estimator. This system facilitates human–robot interaction of a unicycle-modeled mobile robot equipped with a tilt camera. Based on a novel dual-Jacobian visual interaction model, a robust visual tracking controller is proposed to track a dynamic moving target. The proposed controller not only possesses some degree of robustness against the system model uncertainties, but also tracks the target without its 3D velocity information. The visual state estimator aims to estimate the optimal system state and target image velocity, which is used by the visual tracking controller. To achieve this, a self-tuning Kalman filter is proposed to estimate interesting parameters and to overcome the temporary occlusion problem. Furthermore, because the proposed method is fully working in the image space, the computational complexity and the sensor/camera modeling errors can be reduced. Experimental results validate the effectiveness of the proposed method, in terms of tracking performance, system convergence, and robustness.

This paper considers on a number of issues that arise when a trainable machine vision system learns directly from humans. We contrast this to the "normal" situation where machine learning (ML) techniques are applied to a "cleaned" data set which is considered to be perfectly labeled with complete accuracy. This paper is done within the context of a generic system for the visual surface inspection of manufactured parts; however, the issues treated are relevant not only to wider computer vision applications such as medical image screening but also to classification more generally. Many of the issues we consider arise from the nature of humans themselves: They will be not only internally inconsistent but also will often not be completely confident about their decisions, particularly if they are making decisions rapidly. People will also often differ systematically from each other in the decisions they make. Other issues may arise from the nature of the process, which may require the ML to have the capacity for real-time online adaptation in response to users' input. Because of this, it may be that the users cannot always provide input to a consistent level of detail. We describe how all of these issues may be tackled within a coherent methodology. By using a range of classifiers trained on data sets from a compact disc imprint production process, we present results which demonstrate that training methods designed to take proper consideration of these issues may actually lead to improved performance.

In this paper we present further results of our asynchronous and non-invasive BMI for the continuous control of an intelligent wheelchair. Three subjects participated in two experiments where they steered the wheelchair spontaneously, without any external cue. To do so the users learn to voluntary modulate EEG oscillatory rhythms by executing three mental tasks (i.e., mental imagery) that are associated to different steering commands. Importantly, we implement shared control techniques between the BMI and the intelligent wheelchair to assist the subject in the driving task. The results show that the three subjects could achieve a significant level of mental control, even if far from optimal, to drive an intelligent wheelchair.

This paper presents a novel design of visual state estimation for an image-based tracking control system to estimate system state during visual tracking control process. The advantage of this design is that it can estimate the target status and target image velocity without using the knowledge of target's 3D motion-model information. This advantage is helpful for real-time visual tracking controller design. In order to increase the robustness against random observation noise, a neural network based self-tuning algorithm is proposed using echo state network (ESN) technique. The visual state estimator is designed by combining a Kalman filter with the ESN-based self-tuning algorithm. The performance of this estimator design has been evaluated using computer simulation. Several interesting experiments on a mobile robot validate the proposed algorithms.

In this paper we present a novel image classification framework, which is able to automatically re-configure and adapt its featuredriven classifiers and improve its performance based on user interaction during on-line processing mode. Special emphasis is placed on the generic applicability of the framework to arbitrary surface inspection systems. The basic components of the framework include: recognition of regions of interest (objects), adaptive feature extraction, dealing with hierarchical information in classification, initial batch training with redundancy deletion and feature selection components, on-line adaptation and refinement of the classifiers based on operators' feedback, and resolving contradictory inputs from several operators by ensembling outputs from different individual classifiers. The paper presents an outline on each of these components and concludes with a thorough discussion of basic and improved off-line and on-line classification results for artificial data sets and real-world images recorded during a CD imprint production process.

This paper presents an extension of current Global Dynamic Window approaches to arbitrarily shaped holonomic and non-holonomic mobile robots. The algorithm proceeds in two stages. In order to account for an arbitrary robot cross section, the first stage takes the robot’s orientation explicitly into account by constructing a navigation function in the (x, y, θ) configuration space. In a second stage, an admissible velocity is chosen from a window around the robot’s current velocity, which contains all velocities that can be reached under the acceleration constraints. Fast computation over large areas is achieved by adopting multi-resolution (x, y) and (x, y, θ) planning. Several measures are taken to obtain safe and robust robot behaviour. Experimental results on our wheelchair test platform show the feasibility of the approach.

Many elderly and disabled people experience difficulties when maneuvering an electric wheelchair. Additionally, many users are not prescribed a standard electric wheelchair because they lack the necessary cognitive skills or their impairment prevents them from steering an electric wheelchair safely. In order to expand the potential group of electric wheelchairs users and in order to make wheelchair driving a safer and more comfortable experience, there has long been the claim to equip wheelchairs with some form of intelligent controller assisting in difficult or unsafe situations. It has been observed that every user presents different symptoms defining a specific driving pattern. Therefore, if the user is to be helped and not frustrated, his/her particular driving behavior should be taken into account when assisting him/her. Addressing the previous requirement, this paper presents a two step, user-centered approach to shared wheelchair control for our electric wheelchair Sharioto. The framework continuously estimates the user’s plan explicitly before trying to assist him or her (step 1). Uncertainty on the user signals and on the user’s intention is taken into account using a Bayesian approach. This approach can be tailored to the user’s specific driving characteristics. The actual navigation assistance is performed by a fine motion planner that takes the kinodynamic constraints of the wheelchair into account (step 2). As a case study, we analyze the driving behavior of a user who cannot steer to the left and show that he can drive to the left when he intends to.

Although powered wheelchairs provide a well established solution for severely impaired persons they do not cover all needs regarding mobility of people with impairment. In the course of the EC funded research project MOVEMENT a novel approach for a highly adaptable and modular mobility enhancement system is targeted to cover additional user needs. A system consisting of a robotic platform and several dockable application modules is developed that additionally provides assistance for the driving process itself to the user or even takes over the complete driving autonomously. The project also includes development of new solutions for navigation of mobile robot systems including a "low-cost" sensor system as well as adaptable HMI components. This paper describes the concept and the first prototyping results.

The last years have witnessed a significant increase in the percentage of old and disabled people. Members of this population group very often require extensive help for performing daily tasks like moving around or grasping objects. Unfortunately, assistive technology is not always available to people needing it. For instance, steering a wheelchair can represent an extremely fatiguing or simply impossible task to many elderly or disabled users. Most of the existing assistance platforms try to help users without considering their specific needs. However, driving performance may vary considerably across users due to different pathologies or just due to temporary effects like fatigue. Therefore, we propose in this paper a user adapted shared control approach aimed at helping users in driving a power wheelchair. Adaption to the user is achieved by estimating the user's true intent out of potentially noisy steering signals before assisting him/her. The user's driving performance is explicitly modeled in order to recognize the user's intention or plan together with the uncertainty on it. Safe navigation is achieved by merging the potentially noisy input of the user with fine motion trajectories computed online by a 3D planner. Encouraging results on assisting a user who cannot steer to the left are reported on K.U.Leuven's intelligent wheelchair Sharioto.

Many elderly and physically impaired people experience difficulties when maneuvering a powered wheelchair. In order to ease maneuvering, powered wheelchairs have been equipped with sensors, additional computing power and intelligence by various research groups. This paper presents a Bayesian approach to maneuvering assistance for wheelchair driving, which can be adapted to a specific user. The proposed framework is able to model and estimate even complex user intents, i.e. wheelchair maneuvers that the driver has in mind. Furthermore, it explicitly takes the uncertainty on the user's intent into account. Besides during intent estimation, user-specific properties and uncertainty on the user's intent are incorporated when taking assistive actions, such that assistance is tailored to the user's driving skills. This decision making is modeled as a greedy Partially Observable Markov Decision Process (POMDP). Benefits of this approach are shown using experimental results in simulation and on our wheelchair platform Sharioto.

Reservoir Computing (RC) uses a randomly created recurrent neural network where only a linear readout layer is trained. In this work, RC is used for detecting complex events in autonomous robot navigation. This can be extended to robot localization based solely on sensory information. The robot thus builds an implicit map of the environment without the use of odometry data. These techniques are demonstrated in simulation on several complex and even dynamic environments.

The use of shared control techniques has a profound impact on the performance of a robotic assistant controlled by human brain signals. However, this shared control usually provides assistance to the user in a constant and identical manner each time. Creating an adaptive level of assistance, thereby complementing the user's capabilities at any moment, would be more appropriate. The better the user can do by himself, the less assistance he receives from the shared control system; and vice versa. In order to do this, we need to be able to detect when and in what way the user needs assistance. An appropriate assisting behaviour would then be activated for the time the user requires help, thereby adapting the level of assistance to the specific situation. This paper presents such a system, helping a brain-computer interface (BCI) subject perform goal-directed navigation of a simulated wheelchair in an adaptive manner. Whenever the subject has more difficulties in driving the wheelchair, more assistance will be given. Experimental results of two subjects show that this adaptive shared control increases the task performance. Also, it shows that a subject with a lower BCI performance has more need for extra assistance in difficult situations, such as manoeuvring in a narrow corridor.

The possibility to act upon the surrounding environment without using our human nervous system’s efferent pathways enables a new interaction modality that can boost and speed up the human sensor-effector loop. In recent years, brain-computer interface (BCI) research is exploring many applications in different fields: communication, environmental control, robotics and mobility, and neuroprosthetics [1] [2] [3] [4] [5] [6] [7]. Our work in the MAIA project is focused on developing asynchronous and non-invasive BCI to control robots and wheelchairs [7] [8]. It means that the users control such devices spontaneously and at their own paced, by learning to voluntary control specific electroencephalogram (EEG) features measured from the scalp. To this end, the users learn how to voluntary modulate different oscillatory rhythms by execution of different mental tasks (motor and cognitive). To facilitate this learning process, machine learning techniques are utilized, both to find those subject-specific EEG features that maximize the separability between the patterns generated by executing the mental tasks [9], and to train classifiers that minimize the classification error rates of these subject-specific patterns [7]. Finally, to assist the control task, different levels of intelligence are implemented in the device jointly with shared control techniques between the two interacting agents, the BCI system and the intelligent device [10] [11]. One of the main challenges of a non-invasive BCI based on spontaneous brain activity is the non-stationary nature of the EEG signals. Shenoy and co-workers [12] describe two sources of non-stationarity, namely differences

We propose a method for computing the absolute distances to static obstacles using a single omnidirectional camera. The method is applied to mobile robots. We achieve this without restricting the application to predetermined translations or the use of artificial markers. In contrast to prior work, our method is able to build absolute scale 3D without the need of a known baseline length, traditionally acquired by odometry. Instead we use the ground plane assumption together with the camera system's height to determine the scale factor. Using only one omnidirectional camera our method is cheaper, more informative and more compact than the traditional methods for distance determination, especially when a robot is already equipped with a camera for e.g. navigation. It also provides more information since it determines distances in a 3D space instead of in one plane. The experiments show promising results. The algorithm is indeed capable of determining the distances in meters to features and obstacles and is able to locate all major obstacles in the scene.

The term brain computer interface (BCI) is used to denote a direct communication pathway between the brain and an external device. In this poster we propose a BCI based on Steady State Visual Evoked Potentials (SSVEP). This BCI system attains the maximum theoretical transfer rates and an empirical transfer rate that surpass largely all the systems based on movement imagination. As additional features, the system false positive rate is negligible and with a minimal attention of the subject. This means that the subject can follow a conversation or divide visual attention without compromising the control of the external device. Expected to work for several classes, current version allow sending 4 different commands. Using this system we perform a telepresence experiment where one subject at our lab explored another lab several miles away by controlling in real time a robot through Internet.

In this work we present a novel system for autonomous mobile robot navigation. With only an omnidirectional camera as sensor, this system is able to build automatically and robustly accurate topologically organised environment maps of a complex, natural environment. It can localise itself using such a map at each moment, including both at startup (kidnapped robot) or using knowledge of former localisations. The topological nature of the map is similar to the intuitive maps humans use, is memory-efficient and enables fast and simple path planning towards a specified goal. We developed a real-time visual servoing technique to steer the system along the computed path.A key technology making this all possible is the novel fast wide baseline feature matching, which yields an efficient description of the scene, with a focus on man-made environments.