The device consists of a camera which gives the HO an indirect view of a scene (real world); proprioceptive and exteroceptive sensors allowing the recreating of the 3D geometric database of an environment (virtual world). The virtual world is projected onto a video display terminal (VDT). Computer-generated and video images are superimposed. The man-machine interface functions deal mainly with on line building of graphic aids to improve perception, updating the geometric database of the robotic site, and video control of the robot. The superimposition of the real and virtual worlds is carried out through a calibration of the multisensor system. First, the sensing system and several methods used for modeling an environment\'s geometric database in telerobotics have been presented. The operator models unknown objects (of cylindrical or polyhedral pattern) using a video camera(VC) image and a range finder(RF). When the quality of images is poor, the range finder, which is mounted on a site and azimuth rotation turret, brings an indispensable complement by measures of depth. The improvement offered by the cooperation VC/RF has been shown for small depths (below 1.6m). In some particular cases we propose to achieve the fusion of multisensor redundant data, in order to reduce the uncertainty. Then the updating of environment's 3D geometric database in telerobotics by single camera view is presented. It concerns the pose determination of known objects using 2D clues obtained through video images. A two step algorithm is implemented and assessed. The first step is available in case of object large motion, it is a geometric algorithm, which doesn\'t use redundant data; .it provides a first estimate of the pose. The second step is available in case of object small motion. A linear approach is carried out to determine the pose. It allows the use of redundant data. so it improves the accuracy achieved after the first step. KEY WORDS: Teleoperation, Telerobotics, Range finder; Video camera; Sensor modeling; 3D database updating, Image matching. Global Journal of Pure and Applied Sciences Vol.11(1) 2005:153-163

According to the problem to be dealt with, the use of data acquisition system requires the knowledge of several models. In the case of the laser range-finder mounted on a site and azimuth turret, the geometrical model defines the co-ordinate transformation between measurement and reference frames. The paper shows the interest of a global calibration in comparison with a more classical approach which divides the problem into an internal model and external one. During the acquisition process another model called 'physical model' may be useful. It takes into account the fact the laser beam impact is not at a point and introduces a spatial integration. The so-called footprint effect limits the lateral resolution of the range finder at depth discontinuities. In order to correct that effect, an inverse physical model, based on neural networks, is proposed. The improvement of the footprint correction is then evaluated for a 2D and 3D scanning.

This paper presents an augmented reality system that has been developed to reconstruct 3D scenes from a single camera's view. The camera is supposed to be calibrated in the world frame of reference. The 3D-model is known and correctly matched to its 2D-image. A 3D-object pose recovery algorithm that combines three methods to reach a better robustness and accuracy has been developed. A comparison between these methods is given, as well as a discussion about their advantages and drawbacks. Some experimental results on real images that demonstrate the robustness and the accuracy of the proposed system are presented as well as the visual command of a robot based upon the reconstructed scene.

A multi-configuration stereoscopic vision-system is described. We present the mechanical and electronic features of the stereoscopic head together with the hardware. We describe the methods of image processing implemented for overcoming the optical quality drawback of the low-cost CCD mini-cameras. Algorithms and results for the high-level image representation and 3D reconstruction are presented. The structural characteristics, the low-cost and the precision of the stereo system built, allow its utilisation in domestic mobile robotic applications.

MCIT (Multimedia Control Interface for Telerobotics) is an augmented reality system we are working on in our lab. MCIT sensing devices consist of a camera, allowing the operator an indirect view of a scene (real world), proprioceptive and exteroceptive sensors, allowing the building and updating of the 3D geometric database (3D DB) of the environment (virtual world). The virtual world is projected onto a video display terminal (VDT) Computer-generated and video images are superimposed. The man-machine interface functions deal mainly with on-line building of graphics aids to improve perception, updating the geometric database of the task site and video control of the robot. It allows the operator to perform a task level control of the robot. The superimposition of the real and virtual worlds is carried out through a calibration of the multi-sensor system. Within this system there is a direct connection between the video image of an object and its model in the geometric 3D DB. So the robot device can be controlled through the VDT. When the human operator points out an object onto the VDT, in actual fact he points it out in the 3D DB. This allows the programming of the robot with the video image of the robot's site; a task level video control of the robot. In order to improve the accuracy of the building and updating of the 3D geometric database, which are very important for the robot control, we present in this paper an accurate technique for automatic camera calibration and the results for two kinds of cameras

The paper deals with the modelling of unknown objects and location determination of known ones. The operator models objects using data issued from the video image and a time of flight infra red range finder. We present methods to build an embodying volume for unknown objects. These methods are available for polyhedral and conical objects, which represent the basis of most industrial environments. The location determination of known objects is determined using 2D clues obtained through video image. A two-step algorithm is implemented and assessed. The first step is available in case of object large motion, it is a geometric algorithm, which doesn't use redundant data; it provides a first estimate of the pose. The second step is available in case of object small motion. A linear approach is carried out to determine the pose. It allows the use of redundant data, so it improves the accuracy achieved after the first step.

The geometric modeling of the environment can be carried out by incorporating a video camera range finder. When the quality of images is poor, the range finder, which is mounted on a site and azimuth rotation turret, brings an indispensable complement by measures of depth. The purpose of this paper is to build a geometric database of a three-dimensional (3-D) world in telerobotic applications. The...

To achieve the updating of a 3D environment database, the human operator must be provided with methods for recovering the position-orientation of a known object. In this paper two different methods are presented: the first one is based on the edges and uses geometric tranformations to retrieve the position-orientation of the object. The other technique is numerical and based on the vertices and constraints on these vertices. Then a comparison is made between the methods, which focuses on both the application fields and the results given by their respective algorithms. These 3D recovering methods allow to control a robot by pointing out the object to be manipulated onto the video screen.

Active vision tasks (such as tracking or saccadic moves) require gaze control which, in turn, depends on both the optical system and the mechanical structure. The knowledge of the image-joint jacobian is not straightforward and results tra- ditionally from calibration procedures. Physiological studies have shown a real- time adaptation in humans' visual system, that can be modelled by a feedback- error-learning (FEL) scheme. In this paper, we propose a FEL-based modular control system that makes a gaze-platform learn its control while tracking moving objects. The framework used to build the learning system is described along with experiments on ESCHeR - Etl Stereo Compact Head for Robot vision -. As far as the authors are aware, this is the first implementation of a real-time learning con- troller on an active vision system.

Inspired by the properties of the human visual system, a new active vision system called ESCHeR (Etl Stereo Compact Head For Robot Vision) has been recently implemented with foveated wide angle lenses. The lenses exhibit a wide field of view along with a space-varying resolution for facilitating both detection and close observation. However, to handle such optical properties and achieve basic eye movement functions, new calibration methods are needed. Therefore, two novel and online techniques are presented that in one case perform a global identification of the optical process through artificial neural techniques and in the other case compute the physical parameters by using environmental feature-tracking and controlled rotations of the cameras. Self-alignment of the cameras is also achieved using a similar technique

The paper deals with tools a human operator (HO) can use to complete, on-line, the 3D geometric database of the remote environment, even in case of bad condition of vision. The HO provides the system with the 3D geometric primitive of the object's model and the 2D data from the video image. The sensing system: a monocular video camera and a time of flight IR range finder, provides the system with figures relating to the 3D primitives (location, pose, size). Methods for location and pose determination of known objects and for modelling of unknown ones are presented and assessed. These methods are available for polyhedral and cylinder shaped objects, which represent the bases of most of industrial environments.

Most of the techniques proposed for camera calibration consist of identifying a set of physical parameters (focal length, optical center position, coefficient of radial distortion). The determination and inversion of such a model in the case of strong or unstructured distortions is not straightforward. Instead we propose a new approach that uses artificial neural networks' abilities of interpolation and extrapolation to perform the global identification of any optical system, provided a set of pairs of calibration points given in a camera-centered frame of reference and their corresponding image in the image plane are available. A methodological framework is given that deals with the architecture of the neural network the training algorithm and the choice of the learning database. Validity and robustness of the method are shown by comparing reconstruction errors obtained by the neural model, by a theoretical prediction and by classical estimations on both linear and nonlinear lenses.

The paper deals with the updating of environment's geometric database in telerobotics. It concerns the pose determination of known objects using 2D clues obtained through video images. A two step algorithm is implemented and assessed. The first step is available in case of object large motion, it is a geometric algorithm, which doesn't use redundant data; it provides a first estimate of the pose. The second step is available in case of object small motion. A linear approach is carried out to determine the pose. It allows the use of redundant data, so it improves the accuracy achieved after the first step

This paper presents a stereo active vision system which performs tracking tasks on smoothly moving objects in complex backgrounds. Dynamic control of the vergence angle adapts the horopter geometry to the target position and allows to pick it up easily on the basis of stereoscopic disparity features. We introduce a novel vergence control strategy based on the computation of “virtual horopters” to track a target movement generating rapid changes of disparity. The control strategy is implemented on a binocular head, whose right and left pan angles are controlled independently. Experimental results of gaze holding on a smoothly moving target translating and rotating in a complex surrounding demonstrate the efficiency of the tracking system

A method for determinating the objects' position-orientation is described in this paper. The aim of this method is to update an environment's geometric database in telerobotics. When the objects are known, we are able to determine the objects' position-orientation with images provided by the video camera. An algorithm was tested in the case of small variations of objects' position-orientation A simulation was carried out in order to test the accuracy of the algorithm. This simulation tests the limits of the algorithm's accuracy when there is a small rotation in the object's position-orientation. The resistance to noise of the algorithm was also tested. The cases of special configurations of moving objects were established.< >

A trajectory generation module for a mobile robot in a 2D environment and for a robot arm in a 2D1/2 environment made of low height objects placed on a work surface is discussed. For the mobile robot, the problem of driving into a narrow corridor is solved by means of a backtracking process. Obstacle avoidance is carried out by taking into account only the wrist and the end effector of the robot. This allows the determination of a grasping configuration of convex and concave objects, and allows the choice of the shortest, collision-free trajectory

The system described is used in teleoperated robotics to give the human operator a visual aid for the perception of the remote scene and for the robot command in case of indirect viewing via a video camera. The device consists of a camera which gives the operator an indirect view of the real world, sensors allowing the rebuilding of a virtual world. The superimposition of the real and virtual worlds is carried out onto a video display terminal. The system's functions are as follows: building graphics aids for the perception; building a geometric model of the environment; collecting data from the robot site to animate the computer picture of the environment model; matching computer-generated and video images; controlling the robot via the video display terminal.< >

SUMMARY Due to the complexity of teleoperation tasks, human operators figure in the teleoperator perception-decision-control loop. The operator needs an interactive system to handle the huge flow of data between himself and the teleoperator. The scene represented by the robot and its environment is viewed by one or more cameras. However, the video image may be degraded in extreme environments (underwater, space, etc.) or simply inadequate (2-D image). In this paper we describe the visual perception aids based of the scene, and more specifically how these are generated by the method we put forward. The system developed at the LRE superimposes a 3-D synthetic image onto the video picture, and animates the scene in real-time on the basis of sensor information feedback. The graphic image can be generated from models, if the objects are known, otherwise interactively, with the cooperation of the operator if the objects are completely unknown. Experiments show that these graphic aids improve the operator's performance in task execution.

The system described is used in teleoperated robotics to give the human operator a visual aid for the perception of the remote scene when indirect viewing using a video camera. The system enables graphic aids to be super-imposed on the video image. These visual enhancements are based on a 3-D reconstruction of the imaged scene. The computer-generated image of moving objects is animated in real time using sensor data feedback, issued from the task site, measuring the displacements of these objects. Matching of the video and computer-generated images is carried out. The hardware structure consists of VME bus modules. It is a multiprocessor device comprising a master CPU, a data acquisition card and a slave CPU with smart graphics card allowing analogue mixing of the computer-generated and video images. It is a work-oriented graphics card, configured according to the application's specifications.