This paper describes a method for classifying the traversability of terrain by combining unsupervised learning of color models that predict scene geometry with supervised learning of the relationship between geometric features and traversability.A neural network is trained offline on hand-labeled geometric features computed from stereo data.An online process learns the association between color and geometry, enabling the robot to assess the traversability of regions for which there is little range information by estimating the geometry from the color of the scene and passing this to the neural network.This online process is continuous and extremely rapid, which allows for quick adaptations to different lighting conditions and terrain changes.The sensitivity of the traversability judgment is further adjusted online by feedback from the robot's bumper.Terrain assessments from the color classifier are merged with pure geometric classifications in an occupancy grid by computing the intersection of the ray associated with a pixel with a ground plane computed from the stereo range data.We present results from DARPA-conducted tests that demonstrate its effectiveness in a variety of outdoor environments.

We present a system that introduces live captured 3-d content into augmented reality. For each frame we estimate the transformation between the users point of view and a card marker. We then generate a novel or “virtual” view of the live subject from the same perspective. This “virtual view” is generated based on measurements from fifteen cameras that surround the subject. A fast shape-from-silhouette algorithm reconstructs a 3-d model of the live subject from the user’s perspective and re-samples the original images to generate the novel view. This is then rendered into the user’s world at interactive speeds. The result gives the strong impression that the subject is a real part of the 3-D scene. We present a novel mixed reality entertainment application in which users apply tangible computing techniques to interact with a collaborator in a virtual space.

We demonstrate a real-time 3-D augmented reality video-conferencing system. The observer sees the real world from his viewpoint, but modified so that the image of a remote collaborator is rendered into the scene. For each frame, we estimate the transformation between the camera and a fiducial marker using techniques developed in Kato and Billinghurst [1999]. We use a shape-from-silhouette algorithm to generate the appropriate view of the collaborator in real time. This is based on simultaneous measurements from fifteen calibrated cameras that surround the collaborator. The novel view is then superimposed upon the real world image and appropriate directional audio is added. The result gives the strong impression that the virtual collaborator is a real part of the scene.

Interactive robotics demands real-time visual information about the environment. Real time vision processing, however, places a heavy load on the robot’s limited resources, and must accommodate other processes such as speech recognition, animated face displays, communication with other robots, navigation and control. For our entries in the 2000 American Association for Artificial Intelligence robot contest, we developed a vision module capable of providing real-time information about ten or more operators while maintaining at least a 20Hz frame rate and leaving sufficient processor time for the robot’s other capabilities. The vision module uses a probabilistic scheduling algorithm to ensure both timely information flow and a fast frame capture. The vision module makes its information available to other modules in the robot architecture through a shared memory structure. The information provided by the vision module includes the operator information along with a confidence measure and a time stamp. Because of this design, our robots are able to react in a timely manner to a wide variety of visual events.

This article describes the winning entries in the 2000 American Association for Artificial Intelligence Mobile Robot Competition. The robots, developed by Swarthmore College, all used a modular hybrid architecture designed to enable reflexive responses to perceptual input. Within this architecture, the robots integrated visual sensing, speech synthesis and recognition, the display of an animated face, navigation, and interrobot communication. In the Hers d'Oeuvres, Anyone? event, a team of robots entertained the crowd while they interactively served cookies; and in the Urban-Search-and-Rescue event, a single robot autonomously explored a section of the test area, identified interesting features, built an annotated map, and exited the test area within the allotted time.