This paper discusses our past and present experience with operating Mars surface robotics missions. We review current framework and methodology for the Mars 2020 (Perseverance) rover surface operation as well as outline our current development and methodology to create a framework which can be easily adapted to new types of spacecraft. Finally, we discuss how our test platform, “Scarecrow”, which has been adapted to help us develop and test our new showcase for future space robotics missions.

This paper includes a summary, lessons learned, and upcoming plans from the first 210 Mars solar days (sols) of NASA's Mars 2020 Perseverance rover mission. The focus of the paper is on Robotic Operations, which is the team with the primary responsibility for strategic planning, uplink commanding and downlink analysis for rover mobility and navigation, robotic arm operation, the sampling and caching capability including coring, the adaptive caching assembly and the 2nd sample handling robotic arm, and the interface to the Mars helicopter Ingenuity. As of Sol 210 the rover has driven 2663.65 meters, executed 20764 robotic arm and sampling commands, and has successfully completed 13 helicopter flights covering 2382 meters horizontal distance. It includes Operations Readiness Tests executed in preparation for landing, landing and initial checkouts, strategic route planning to the science destination and waypoints, and surface checkout of all of the robotics capabilities of the rover. It also discusses the strategic planning and tactical agility needed to interleave science investigations and the technology demonstration of the Mars helicopter flights where a minimum distance had to be maintained between the rover and helicopter during flights. It discusses the challenges with planning robotic operations and addressing anomalies with the larger uncertainty present during early mission operations. It also discusses the impact on robotic operations from lessons incorporated from previous missions.

The NASA Ingenuity helicopter and Perseverance rover demonstrated the first coordinated operation between a ground based rover and aerial helicopter on another planet in April 2021. This included successful deployment of the helicopter from the belly of the rover and multiple flights of the helicopter while the rover documented the event with imaging, video, and audio. The rover relayed commands and wake up schedules to the helicopter, which it received during its daily communications with the Deep Space Network. Thus, operators on Earth needed to include commanding for both the rover and the helicopter in the daily uplink bundles. Sequencing and visualization of spacecraft commands is an integral part of verifying a set of activities prior to uplink, through simulation of hardware motion and interaction with terrain meshes. The introduction of the Ingenuity helicopter presented a need for planning multi-spacecraft interactions between the rover and helicopter. This was especially important during the critical event of driving away post helicopter deployment, involving close clearances between the systems. Helicopter flights needed to ensure a keepout distance from the rover, subsequent rover traverses needed to avoid coming into contact with the helicopter, and imaging needed to be correctly pointed to include the helicopter location and flight path. All of these capabilities were incorporated into the operations planning software used by the Helicopter Integration Engineer team to model and verify plans involving both rover and helicopter commanding. In this paper we discuss updates to the Robot Sequencing and Visualization Program (RSVP), used for coordinating the interactions between the Ingenuity helicopter and Perseverance rover. RSVP is a visual simulation tool that fuses CAD models of the rover and helicopter, polygonal terrain meshes derived from stereo imagery, orbital and local digital elevation models (DEM), and a high fidelity kinematic simulation of vehicle/terrain interaction that is driven by a version of the Perseverance flight software. It allows for detailed simulation and inspection of all robotic commanding by operators prior to uplink to the rover on Mars. We discuss the features that were implemented specific to coordinated rover and helicopter commanding, the testing and validation on Earth in operations readiness tests, as well as the ultimate usage for Ingenuity helicopter deployment, imaging, and flights on Mars.

NASA's Jet Propulsion Laboratory has built and operated four rovers on the surface of Mars. Two and three dimensional visualization has been extensively employed to command both the mobility and robotic arm operations of these rovers. Stereo visualization has been an important component in this set of visualization techniques. This paper discusses the progression of the implementation and use of visualization techniques for in-situ operations of these robotic missions. Illustrative examples will be drawn from the results of using these techniques over more than ten years of surface operations on Mars.

The Rover Sequencing and Visualization Program (RSVP) is a tool suite used for building command sequences for the Mars Science Laboratory rover Curiosity. RSVP was previously used for other missions and for in-house research projects and proposal efforts. RSVP has undergone extensive modifications and enhancements over previous versions in order to support more challenging requirements and to make it more adaptable for future missions. This paper will provide a brief overview of many of the specific enhancements made for the MSL mission.

This paper describes a high fidelity mission concept systems testbed at JPL, called Lunar Surface Operations Testbed (LSOT). LSOT provides a unique infrastructure that enables mission concept studies designers to configure and demonstrate end-to-end surface operations using existing JPL mission operations and ground support tools, Lander, robotic arm, stereo cameras, flight software, and soil simulant (regolith), in a high fidelity functional testbed. This paper will describe how LSOT was used to support the MoonRise mission concept study. MoonRise: Lunar South Pole-Aitken Basin Sample Return Mission would place a lander in a broad basin near the moon's South Pole and return approximately two pounds of lunar materials to Earth for study. MoonRise was one of three candidate missions competing to be selected as the third mission for NASA's New Frontiers Program of Solar System Explorations. LSOT was used to demonstrate JPL's extensive experience and understanding of the MoonRise Lander capabilities, design maturity, surface operations systems engineering issues, risks and challenges.

Mars Exploration Rover Opportunity Terramechanics Across Ripple Covered Bedrock In Meridiani Planum

The Rover Sequencing and Visualization Program is a suite of tools for the commanding of planetary rovers and other robots. The overall program architecture is based on components synchronizing and messaging over an inter-process communication bus. This paper will provide a description of these components and discuss how they work together to provide several modes of information display and visualization for robotic commanding. Results from over 3000 sols of combined operations of the Mars Rovers Spirit and Opportunity will be used to illustrate features of the RSVP suite. Support for the Phoenix Mars Lander, Mars Science Lander and other lunar and planetary missions will also be discussed.

On January 24, 2004, the Mars Exploration Rover named Opportunity successfully landed in the region of Mars known as Meridiani Planum, a vast plain dotted with craters where orbiting spacecraft had detected the signatures of minerals believed to have formed in liquid water. The first pictures back from Opportunity revealed that the rover had landed in a crater roughly 20 meters in diameter - the only sizeable crater within hundreds of meters - which became known as Eagle Crater. And in the walls of this crater just meters away was the bedrock MER scientists had been hoping to find, which would ultimately prove that this region of Mars did indeed have a watery past. Opportunity explored Eagle Crater for almost two months, then drove more than 700 meters in one month to its next destination, the much larger Endurance Crater. After surveying the outside of Endurance Crater, Opportunity drove into the crater and meticulously studied it for six months. Then it went to examine the heat shield that had protected Opportunity during its descent through the Martian atmosphere. More than a year since landing, Opportunity is still going strong and is currently en route to Victoria Crater - more than six kilometers from Endurance Crater. Opportunity has driven more than four kilometers, examined more than eighty patches of rock and soil with instruments on the robotic arm, excavated four trenches for subsurface sampling, and sent back well over thirty thousand images of Mars - ranging from grand panoramas to up close microscopic views. This paper details the experience of driving Opportunity through this alien landscape from the point of view of the Rover Planners, the people who tell the rover where to drive and how to use its robotic arm.

Spirit is one of two rovers that landed on Mars in January 2004 as part of NASA's Mars Exploration Rover mission. As of July 2005, Spirit has traveled over 4.5 kilometers across the Martian surface while investigating rocks and soils, digging trenches to examine subsurface materials, and climbing hills to reach outcrops of bedrock. Originally designed to last 90 sols (Martian days), Spirit has survived over 500 sols of operation and continues to explore. During the mission, we achieved increases in efficiency, accuracy, and traverse capability through increasingly complex command sequences, growing experience, and updates to the on-board and ground-based software. Safe and precise mobility on slopes and in the presence of obstacles has been a primary factor in development of new software and techniques.

The Rover Sequencing and Visualization Program is a suite of tools for the commanding of planetary rovers which are subject to significant light time delay and thus are unsuitable for tele-operation. The two main components of the program are the Rover Sequence Editor and HyperDrive. This paper focuses on HyperDrive, the immersive visualization component of the system. HyperDrive fuses multiple data types returned from the vehicle in order to facilitate an operator understanding of the current environment and past rover performance, so that safe effective command sequences for successful future rover activities may be generated on a tight tactical timeline. Multiple display and task specific interaction modalities are provided to most efficiently present relevant spatial and time series data to the sequence builder

Operating a rover on Mars is not possible using teleoperations due to the distance involved and the bandwith limitations. To operate these rovers requires sophisticated tools to make operators knowledgeable of the terrain, hazards, features of interest, and rover state and limitations, and to support building command sequences and rehearsing expected operations. This paper discusses how the Rover Sequencing and Visualization program and a small set of associated tools support this requirement.

The Rover Sequencing and Visualization Program (RSVP) is a suite of applications for composing, visualizing, and simulating sequences of spacecraft commands. The successor to the Rover Control Workstation software used in JPL's highly successful 1997 Mars Pathfinder mission, RSVP provides fast command editing and highly accurate simulation for Mars rover missions. Though it shares some conceptual and architectural similarities with its 1997 predecessor, RSVP was implemented from scratch for JPL's 2003-04 Mars Exploration Rover (MER) missions. RSVP is the software used to “drive” the MER rovers on the Martian surface, and was also used to command them during their cruise phase.

Rovers don't run Linux yet, but back on Earth, Linux is the platform of choice for planning their routes and collecting data.

Current developments in immersive environments for mission planning include several tools which make up a system for performing and rehearsing missions. This system, known as the Rover Sequencing and Visualization Program (RSVP), includes tools for planning long range sorties for highly autonomous rovers, tools for planning operations with robotic arms, and advanced tools for visualizing telemetry from remote spacecraft and landers. One of the keys to successful planning of rover activities is knowing what the rover has accomplished to date and understanding the current rover state. RSVP builds on the lessons learned and the heritage of the Mars Pathfinder mission This paper will discuss the tools and methodologies present in the RSVP suite for examining rover state, reviewing previous activities, visually comparing telemetered results to rehearsed results, and reviewing science and engineering imagery. In addition we will present how this tool suite was used on the Mars Exploration Rovers (MER) project to explore the surface of Mars.

The sequence rehearsal tool in the Rover Sequencing and Visualization Program (RSVP) is based on modeling and simulation of the multi-body mechanical systems. The methodology has been developed to support a real-time interactive graphics mode for the visualization tool, using the Configuration Kinematics (CK) and 3D terrain models. The sequence simulation is carried out using the on-board flight software modules for realistic rover behavior. It enables the scientists and rover planners to effectively develop the command sequences for rover safety and maximize the science returns of the Mars Exploration Rover (MER) mission. This paper describes the innovative numerical algorithm and the command sequence simulation of the MER mission for surface operations. Recently, the Rover Analysis Modeling Simulation algorithm ( 13, which computes the configuration of the robotic vehicles on rough terrain, has been developed at JPL for planetary surface exploration rover missions. Based on these and related methods, we've developed the sequence simulation tool to achieve the required accuracy and support the time-critical applications. One crucial design decision of the command sequence simulation tool is to represent the multi-body model as a collection of subsystems. Not only is the rover model partitioned by an object-oriented hierarchy of multi-body subsystems, but the numerical algorithms that carry out the solutions of configuration kinematics and inverse kinematics are also implemented via object-oriented methodology. Inheriting the subsystem model and the numerical methods are the structure-preserving simulators, which directly map the numerical solutions to the corresponding physical structures. The subsystem simulators constitute the sequence rehearsal engine that is advantageous to traditional multi-body simulations. The benefits of the structure preserving design for command sequence simulation will be detailed in this paper. 1. Modeling of the Mars Exploration Rovers