NASA's Jet Propulsion Laboratory has built and operated four rovers on the surface of Mars. Two and three dimensional visualization has been extensively employed to command both the mobility and robotic arm operations of these rovers. Stereo visualization has been an important component in this set of visualization techniques. This paper discusses the progression of the implementation and use of visualization techniques for in-situ operations of these robotic missions. Illustrative examples will be drawn from the results of using these techniques over more than ten years of surface operations on Mars.

The Rover Sequencing and Visualization Program (RSVP) is a tool suite used for building command sequences for the Mars Science Laboratory rover Curiosity. RSVP was previously used for other missions and for in-house research projects and proposal efforts. RSVP has undergone extensive modifications and enhancements over previous versions in order to support more challenging requirements and to make it more adaptable for future missions. This paper will provide a brief overview of many of the specific enhancements made for the MSL mission.

This paper describes a high fidelity mission concept systems testbed at JPL, called Lunar Surface Operations Testbed (LSOT). LSOT provides a unique infrastructure that enables mission concept studies designers to configure and demonstrate end-to-end surface operations using existing JPL mission operations and ground support tools, Lander, robotic arm, stereo cameras, flight software, and soil simulant (regolith), in a high fidelity functional testbed. This paper will describe how LSOT was used to support the MoonRise mission concept study. MoonRise: Lunar South Pole-Aitken Basin Sample Return Mission would place a lander in a broad basin near the moon's South Pole and return approximately two pounds of lunar materials to Earth for study. MoonRise was one of three candidate missions competing to be selected as the third mission for NASA's New Frontiers Program of Solar System Explorations. LSOT was used to demonstrate JPL's extensive experience and understanding of the MoonRise Lander capabilities, design maturity, surface operations systems engineering issues, risks and challenges.

The Rover Sequencing and Visualization Program is a suite of tools for the commanding of planetary rovers and other robots. The overall program architecture is based on components synchronizing and messaging over an inter-process communication bus. This paper will provide a description of these components and discuss how they work together to provide several modes of information display and visualization for robotic commanding. Results from over 3000 sols of combined operations of the Mars Rovers Spirit and Opportunity will be used to illustrate features of the RSVP suite. Support for the Phoenix Mars Lander, Mars Science Lander and other lunar and planetary missions will also be discussed.

Monitoring of spacecraft and robotic activities, both in realtime and after the fact, can benefit from the use of three-dimensional visualization tools and spacecraft models. The simple case of animating a robot model based on a telemetry stream provides an intuitive understanding of the robot's activities that far exceeds that available through dials, graphs, and other common widgets. The addition of such widgets into the three-dimensional environment, in conjunction with the robot model, enhances understanding while focusing attention on critical areas. The design and architecture of visualization tools for analyzing spacecraft state were developed to facilitate the association of telemetered data channels to the three-dimensional model of the spacecraft or robot. In addition, the development of custom indicators for any data channel helps to produce a tool geared to maximizing the understanding of the spacecrafts state. Indicators can be enabled or disabled to allow an operator to locus on key data values for a specific activity. In addition to the three-dimensional visualization capabilities, standard graphing and plotting tools allow review of telemetry histories and playback of histories through the animation within the 3D environment. These capabilities allow for rapid assimilation of spacecraft activities and state. In addition to telemetry histories, the results of pre-activity simulations can be visualized and then visually compared to actual telemetry to verify correct performance. For in-situ missions, models of the local terrain can be included and interactions between the robot and the terrain analyzed. The three-dimensional capabilities presented enhance the understanding of robotic and spacecraft activities and state to improve operations, reliability, and safety.

The rover sequencing and visualization program (RSVP) suite of tools has been a critical factor in the success of the Mars exploration rover (MER) missions. It would be impossible to prepare the large command loads each sol without the capabilities that it possesses. It has proven to be robust and easy to use and capable of answering key questions about sequence validity and constraints. Certainly, training is required to use RSVP, but this is primarily in the general area of command sequencing and rover operations. Once these concepts are understood, RSVP feels natural for building sequences. RSVP has met its prime requirements of supporting rapid assimilation and understanding of the terrain and operational constraints, rapid sequence generation and validation, and production of documentation and archival products. This can be seen in the very limited number of sols lost due to errors in the command sequences. The success of the MER mission and the tremendous amount of science data collected attest to the capability of RSVP

On January 24, 2004, the Mars Exploration Rover named Opportunity successfully landed in the region of Mars known as Meridiani Planum, a vast plain dotted with craters where orbiting spacecraft had detected the signatures of minerals believed to have formed in liquid water. The first pictures back from Opportunity revealed that the rover had landed in a crater roughly 20 meters in diameter - the only sizeable crater within hundreds of meters - which became known as Eagle Crater. And in the walls of this crater just meters away was the bedrock MER scientists had been hoping to find, which would ultimately prove that this region of Mars did indeed have a watery past. Opportunity explored Eagle Crater for almost two months, then drove more than 700 meters in one month to its next destination, the much larger Endurance Crater. After surveying the outside of Endurance Crater, Opportunity drove into the crater and meticulously studied it for six months. Then it went to examine the heat shield that had protected Opportunity during its descent through the Martian atmosphere. More than a year since landing, Opportunity is still going strong and is currently en route to Victoria Crater - more than six kilometers from Endurance Crater. Opportunity has driven more than four kilometers, examined more than eighty patches of rock and soil with instruments on the robotic arm, excavated four trenches for subsurface sampling, and sent back well over thirty thousand images of Mars - ranging from grand panoramas to up close microscopic views. This paper details the experience of driving Opportunity through this alien landscape from the point of view of the Rover Planners, the people who tell the rover where to drive and how to use its robotic arm.

The Mars Exploration Rover (MER) mission has returned tremendous scientific information on a daily basis, owing to the efficient sequencing capability of the ground system tools. For planning the mobility and instrument deployment device (IDD) sequences, physical-based simulation is applied to achieve fast and effective sequencing of complex rover and IDD maneuvers. The sequence rehearsal tool of the Rover Sequencing and Visualization Program (RSVP) is based on modeling and simulation of the multi-body mechanical systems. Using configuration kinematics (CK) and 3D terrain models, a methodology was developed to support a real-time, interactive graphics mode for the visualization tool. The sequence simulation is carried out using the on-board flight software modules for realistic rover behavior. This enables the scientists and rover planners to effectively develop the command sequences to maximize the science return of the MER mission while maintaining rover safety. This paper describes the innovative numerical algorithms and the command sequence simulation used by the MER mission for planning surface operations.

Spirit is one of two rovers that landed on Mars in January 2004 as part of NASA's Mars Exploration Rover mission. As of July 2005, Spirit has traveled over 4.5 kilometers across the Martian surface while investigating rocks and soils, digging trenches to examine subsurface materials, and climbing hills to reach outcrops of bedrock. Originally designed to last 90 sols (Martian days), Spirit has survived over 500 sols of operation and continues to explore. During the mission, we achieved increases in efficiency, accuracy, and traverse capability through increasingly complex command sequences, growing experience, and updates to the on-board and ground-based software. Safe and precise mobility on slopes and in the presence of obstacles has been a primary factor in development of new software and techniques.

Operating a rover on Mars is not possible using teleoperations due to the distance involved and the bandwith limitations. To operate these rovers requires sophisticated tools to make operators knowledgeable of the terrain, hazards, features of interest, and rover state and limitations, and to support building command sequences and rehearsing expected operations. This paper discusses how the Rover Sequencing and Visualization program and a small set of associated tools support this requirement.

Immersive environments are being used to support mission operations at the Jet Propulsion Laboratory. This technology contributed to the Mars Pathfinder Mission in planning sorties for the Sojourner rover and is being used for the Mars Exploration Rover (MER) missions. The stereo imagery captured by the rovers is used to create 3D terrain models, which can be viewed from any angle, to provide a powerful and information rich immersive visualization experience. These technologies contributed heavily to both the mission success and the phenomenal level of public outreach achieved by Mars Pathfinder and MER. This paper will review the utilization of terrain modelling for immersive environments in support of MER..

The Rover Sequencing and Visualization Program (RSVP) is a suite of applications for composing, visualizing, and simulating sequences of spacecraft commands. The successor to the Rover Control Workstation software used in JPL's highly successful 1997 Mars Pathfinder mission, RSVP provides fast command editing and highly accurate simulation for Mars rover missions. Though it shares some conceptual and architectural similarities with its 1997 predecessor, RSVP was implemented from scratch for JPL's 2003-04 Mars Exploration Rover (MER) missions. RSVP is the software used to “drive” the MER rovers on the Martian surface, and was also used to command them during their cruise phase.

Current developments in immersive environments for mission planning include several tools which make up a system for performing and rehearsing missions. This system, known as the Rover Sequencing and Visualization Program (RSVP), includes tools for planning long range sorties for highly autonomous rovers, tools for planning operations with robotic arms, and advanced tools for visualizing telemetry from remote spacecraft and landers. One of the keys to successful planning of rover activities is knowing what the rover has accomplished to date and understanding the current rover state. RSVP builds on the lessons learned and the heritage of the Mars Pathfinder mission This paper will discuss the tools and methodologies present in the RSVP suite for examining rover state, reviewing previous activities, visually comparing telemetered results to rehearsed results, and reviewing science and engineering imagery. In addition we will present how this tool suite was used on the Mars Exploration Rovers (MER) project to explore the surface of Mars.

One goal for future Mars missions is to navigate a rover to science targets not visible to the rover, but seen in orbital or descent images. In order to support and improve long-range navigation capabilities, we generate 3D terrain maps using all available images, including surface images from the lander and/or rover, descent images from the lander, and orbital images from current and future Mars orbiters. The techniques used include wide-baseline stereo mapping for terrain distant from the rover, bundle adjustment for high-accuracy mapping of surface images, and structure-from-motion techniques for mapping using descent and orbital images. The terrain maps are compiled using a system for unifying multi-resolution models and integrating three-dimensional terrains.

The Rover Sequencing and Visualization Program (RSVP) is a suite of tools for sequencing of planetary rovers, which are subject to significant light time delay and thus are unsuitable for teleoperation.

Visualization tools play a key role in the exploration of outer space. Since it is difficult and expensive to send humans to other planets, immersive visualization of such hostile environments is as close as we will get for some time. Visualization is also used in a variety of supporting roles for deep space missions, from simulation and rehearsal of planned operations to analysis of spacecraft state to analysis of science data returned from a variety of instruments. The panelists will discuss their experiences in collecting data in deep space, transmitting it to Earth, processing and visualizing it here, and using the visualization to drive the continued mission. This closes the loop, making missions more responsive to their environment, particularly in-situ operations on planetary surfaces and within planetary atmospheres.

The Tropospheric Emission Spectrometer (TES) instrument seeks to analyze the chemical composition of the atmosphere based on the emission, absorption, and transmission of infrared radiation. Meeting the scientific objectives requires demanding analysis of the data being collected and processed. Visualization tools will assist in the understanding of the data and of the effects of the various types of processing being performed. The TES visualization tools are designed to verify correct functioning; of the instrument, provide early detection of potential problems, and report on the quality and validity of the science data for drawing scientific conclusions. Visualization displays include the Level III tools for displaying the end result of all the processing, merged and georeferenced for display relative to maps or global images, and displays for characterizing the behavior of the science processing algorithms and exploring the effects of implementation decisions. Displays of interest include plots of spectra and profiles, animations showing variations in the data along spatial or temporal axes, and results of various operations on the data. Together, these tools provide a visualization suite for more rapidly analyzing the science results of the TES instrument and detecting and identifying problems in the instrument or processing system