In this paper, we achieve reliable, sub-second motion planning to Cartesian end-effector targets in changing real-world environments for a high degree-of-freedom $(\mathbf{12} + \mathbf{7}$ DoF), redundant robot. Several recent works have tackled a subset of these challenges, leveraging innovations in optimization, deep learning, and/or intelligent sampling. However, deployment of robots in real-world environments demands reliably tackling all three challenges: changing environments, fast planning, and high DoF robots. In this work, we leverage (1) large-scale Dynamic Roadmaps (DRM) enabled through GPU-accelerated collision-checking, combined with (2) an optimization-based local inverse kinematics (IK) solver. The high-level principle behind our approach is to offload as much computation either offline or onto the GPU as possible in order to simplify and speed up online planning. Even in changing environments, this enables fast planning in configuration space to a neighborhood of a target pose(s) specified in Cartesian space, whereby the final connection to the target pose is made via a local IK solver. We ran several experiments in an unmodified real-world grocery store with this motion planner on a $\mathbf{12} + \mathbf{7}$ DoF mobile manipulation robot executing a grocery fulfillment task, achieving ≈0.3s average planning times with 100% success rate across 950 motion plans.

The development of the Sampling and Caching Subsystem (SCS) of the Mars 2020 Rover Mission is highly dependent on testing of prototype hardware and software operating in explicit conditions as part of integrated testbeds. To achieve relevant integration of hardware and software while maintaining rapid algorithm development capabilities and high testing throughput, the Controls and Autonomy for Sample Acquisition and Handling (CASAH) software system was developed. CASAH is an implementation of the Intelligent Robotics System Architecture (IRSA), which mimics JPL Flight Software (FSW) in that it is divided into modules that run separate processes that communicate via message passing, each module is assigned an owner that is a single developer, and the operator initiates requests via a text-based interface that interprets sequences of commands. IRSA enables a modular breakdown of CASAH that follows that of 2020 Flight Software, so developers can take an algorithm from a module in CASAH and re-code it into the same module in FSW. As deployment of CASAH has grown to ten testbeds - each with different hardware and objectives - bottom-up design decisions have been intentionally made to keep the system lightweight and maintainable by a very small team. To date, CASAH has been used to run 1393 different tests. This work describes CASAH, the testbeds and functionality it supports, the tools used to manage the development and sharing of code, and the features of the software. Lessons learned over the past three years of development and deployment are provided.

A sample acquisition and caching architecture has been developed for the proposed Mars 2020 mission that acquires samples directly into sample tubes in coring bits. Sample tubes are inserted and removed from the bits, seals are inserted into the tubes, and the tubes are placed in the sample canister. This paper describes initial algorithms, implementation, and test results where sample tube transfer operations are performed using the sampling arm where the tube gripper is on the arm turret with the sampling tool. The testbed utilizes a robotic arm with harmonic drives in the actuators which then provide an arm with low backlash and high resolution of motion. A six-axis force-torque sensor provides force feedback for the tube transfer operations. The results show that the tube transfer operations can be done robustly using the existing testbed robotic arm and caching hardware. Future work is described which will investigate the effects on robustness of the tube transfer with reduced capability robotic arm and force sensing.

This paper describes the development of hardware, software, and algorithms for a prototype limbed robot capable of surface mobility on small bodies (asteroids and comets). It also describes the development of a laboratory testbed capable of simulating the micro-gravity and terrain of small bodies. A path following algorithm that uses visual odometry, robot body kinematics, and a variety of specialized gaits was used to demonstrate micro-gravity mobility with as few as 12 actuators. A mapping algorithm is also demonstrated that will enable path planning, limb trajectory planning, mobile grasping, and foot placement in future work. The results of this paper demonstrate that robust, stable, and precise small body mobility is feasible with a limbed robot.

The design methodologies of using sequence diagrams, multi-process functional flow diagrams, and hierarchical state machines were successfully applied in designing three MSL (Mars Science Laboratory) flight software modules responsible for handling actuator motions of the CHIMRA (Collection and Handling for In situ Martian Rock Analysis), IC (Inlet Covers), and DRT (Dust Removal Tool) mechanisms. The methodologies were essential to specify complex interactions with other modules, support concurrent foreground and background motions, and handle various fault protections. Studying task scenarios with multi-process functional flow diagrams yielded great insight to overall design perspectives. Since the three modules require three different levels of background motion support, the methodologies presented in this paper provide an excellent comparison. All three modules are fully operational in flight.

One of the goals of the Mars Science Laboratory (MSL) mission is to collect powderized samples from the interior of rocks in order to deliver these samples to onboard science instruments. This paper describes the algorithms and software used to control the drill, which is the component of the sample collection and delivery system that directly interacts with rocks to create and acquire powderized samples from their interior. This is the first time that autonomous drilling of rocks has ever been performed on another planet. One of the most important components of the algorithm used for drilling is a force feedback control system used to regulate the force applied to the rock during drilling. This algorithm and all of the other algorithms and software used to enable the process of robustly, efficiently, and autonomously drilling into rocks with a priori unknown and widely varying properties are described in detail in this paper. Results are shown from drilling rocks using the drill software on testbed hardware on Earth as part of the software development process. Results are also shown from the first holes drilled with the flight vehicle on Mars, thus successfully demonstrating the first extraterrestrial autonomous drilling of a rock.

Progress has been made in the development of a family of robotic land vehicles having modular and minimalist design features chosen to impart a combination of robustness, reliability, and versatility. These vehicles at earlier stages of development were described in two previous NASA Tech Briefs articles: Exploratory Robotic Vehicles (NPO-20944), Vol. 25, No. 7 (July 2001), page 56; and More About Reconfigurable Exploratory Robotic Vehicles (NPO-30890), Vol. 33, No. 8 (August 2009), page 40. Conceived for use in exploration of the surfaces of Mars and other remote planets, these vehicles could also be adapted to terrestrial applications, including exploration of volcanic craters or other hostile terrain, military reconnaissance, inspection of hazardous sites, and searching for victims of earthquakes, landslides, avalanches, or mining accidents. In addition, simplified versions of these vehicles might be marketable as toys. The most basic module in this family of reconfigurable robots is the Axel rover, which has a cylindrical body with two main wheels and a trailing link. Inside its body are three motors and associated mechanisms for driving the two wheels and for rotating the link 360 around its symmetrical body. The actuated link serves several purposes: It is used as a lever arm to react to the wheels thrust to move Axel in multiple directions. It is used to rotate the Axel housing in order to tilt, to the desired angle, any sensors and instruments mounted on or in the Axel housing. It provides an alternative mobility mode, which is primarily used in its tethered configuration. Turn ing the link into the ground in lieu of driving the wheels causes the Axel housing and wheels to roll as a unit and thereby leads to a tumbling motion along the ground. With a tether mounted around Axel s cylindrical body, the link serves as a winch mechanism to reel and unreel the tether raising and lowering Axel over steep and vertical surfaces (Figure 1). Sensors, computation, and communication modules are also housed inside Axel s body. A pair of stereo vision cameras provides three-dimensional view for autonomous navigation and avoiding obstacles. Inertial sensors determine the tilt of the robot and are used for estimating its motion. In a fully developed version, power would be supplied by rechargeable batteries aboard Axel; at the time of reporting the information for this article, power was supplied from an external source via a cable. In and of itself, the Axel rover is fully capable of traversing and sampling terrains on planetary surfaces. By use of only the two main wheel actuators and the caster link actuator, Axel can be made to follow an arbitrary path, turn in place, and operate upside- down or right-side-up. If operated in a tethered configuration, as shown in Figure 1, it can be made to move down and up a steep crater wall, descend from an overhang to a cave, and ascend from the cave back to the overhang, all by use of the same three actuators. Such tethered operation could be useful in searching for accident victims or missing persons in mines, caves, and rubble piles. Running the tether through the caster link enhances the stability of Axel and provides a restoring force that keeps the link off the ground for the most part during operation on a steep slope. In its extended configuration, two Axel modules can dock to either side of a payload module to form the four wheeled Axel2 rover (Figure 2). Additional payload and Axel modules can dock to either side of the Axel2 to form the Axel3 rover, extending its payload capacity and its mobility capabilities.

Modular exploratory robotic vehicles that will be able to reconfigure themselves in the field are undergoing development. Proposed for use in exploration of the surfaces of Mars and other remote planets, these vehicles and others of similar design could also be useful for exploring hostile terrain on Earth.

This paper describes the design, implementation, and experimental results of a navigation system for planetary rovers called Terrain Adaptive Navigation (TANav). This system was designed to enable greater access to and more robust operations within terrains of widely varying slippage. The system achieves this goal by using onboard stereo cameras to remotely classify surrounding terrain, predict the slippage of that terrain, and use this information in the planning of a path to the goal. This navigation system consists of several integrated techniques: goodness map generation, terrain triage, terrain classification, remote slip prediction, path planning, High-Fidelity Traversability Analysis, and slip-compensated path following. Results from experiments with an end-to-end onboard implementation of the TANav system in a Mars analog environment are shown and compared to results from experiments with a more traditional navigation system that does not account for terrain properties. © 2009 Wiley Periodicals, Inc.

A sample sealing technique was developed and tested for a possible Mars Sample Return mission application. The effect on the scientific viability of biological samples from storage of samples in a sample container for a long period of time on the Martian surface was also investigated. Sealing techniques were investigated and a soldering concept was developed and tested to provide a hermetic seal between a sample tube and cap. A sample caching subsystem design concept was updated to allow for sealing of sample tubes. A gas-tight vessel was constructed that could be used to simulate environmental conditions that would be experienced by a sample of regolith on Mars and to test the affects of thermal cycling of the vessel on psychrophilic microorganisms embedded in a regolith simulant to assess the degree of deterioration of the microorganisms.

A navigation system designed for a Mars rover has been designed to deal with rough terrain and/or potential slip when evaluating and executing paths. The system also can be used for any off-road, autonomous vehicles. The system enables vehicles to autonomously navigate different terrain challenges including dry river channel systems, putative shorelines, and gullies emanating from canyon walls. Several of the technologies within this innovation increase the navigation system s capabilities compared to earlier rover navigation algorithms.

Technology for coring from a low-mass rover has been developed to enable core sample acquisition where a planetary rover experiences moderate slip during the coring operation. A new stereo vision technique, Absolute Motion Visual Odometry, is used to measure rover slip during coring and the slip is accommodated through corresponding arm pose updating. Coring rate is controlled by feedback of themeasured force of the coring tool against the environment. Test results in the JPL Marsyard show for the first time that coring from a low-mass rover with slip is feasible.

Technology for coring from a low-mass rover has been developed to enable core sample acquisition where a planetary rover experiences moderate slip during the coring operation. A new stereo vision technique, Absolute Motion Visual Odometry, is used to measure rover slip during coring and the slip is accommodated through corresponding arm pose updating. Coring rate is controlled by feedback of the measured force of the coring tool against the environment. Test results in the JPL Marsyard show that coring from a low-mass rover with slip is feasible.

Results from the experimental testing of a navigation system for planetary rovers called Terrain Adaptive Navigation (TANav) are shown here. This system was designed to enable greater access to and more robustoperations in terrains with widely varying slippage.The system achieves this goal by using onboard stereocameras to remotely classify terrain, predict the slippage of that terrain, and use this information in the planning of a path to the goal. An end-to-end onboard demonstration of the system in a Mars analog environment is shown with promising results.

A design and a partially developed application framework were presented for improving localization and targeting for surface spacecraft. The program has value for the Mars Science Laboratory mission, and has been delivered to support the Mars Exploration Rovers as part of the latest version of the Maestro science planning tool. It also has applications for future missions involving either surface-based or low-altitude atmospheric robotic vehicles. The targeting and localization solutions solve the problem of how to integrate localization estimate updates into operational planning tools, operational data product generalizations, and flight software by adding expanded flexibility to flight software, the operations data product pipeline, and operations planning tools based on coordinate frame updates during a planning cycle. When acquiring points of interest (targets) for the rover, instead of using a temporal method for reusing previously acquired targets, this system uses a spatial method to avoid tedious and repetitive target re-designation needed to keep target relevance accurate. Instead of creating a target that is reusable only for a sol (Martian day), the target is defined in a way to make it reusable for a planning position (the vehicle position indicated by a Site and Drive index pair) from which the vehicle will begin a command cycle. This work was done by Mark W. Powell, Thomas Crockett, Jason M. Fox, Joseph C. Joswig, Jeffrey S. Norris, and Kenneth J. Rabe of NASA’s Jet Propulsion Laboratory. The software used in this innovation is available for commercial licensing. Please contact Karina Edmonds of the California Institute of Technology at (626) 395-2322. Refer to NPO-43847.

The vehicles used to explore the Martian surface require a high degree of autonomy to navigate challenging and unknown terrain, investigate targets, and detect scientific events. Increased autonomy will be critical to the success of future missions. In July 1997, as part of NASA's Mars Pathfinder mission, the Sojourner rover became the first spacecraft to autonomously drive on another planet. The twin Mars Exploration Rovers (MER) vehicles landed in January 2004, and after four years Spirit had driven more than four miles and Opportunity more than seven miles-lasting well past their projected three-month lifetime and expected distances traveled. The newest member of the Mars rover family will have the ability to autonomously approach and inspect a target and automatically detect interesting scientific events. In fall 2009, NASA plans to launch the Mars Science Laboratory (MSL) rover, with a primary mission of two years of surface exploration and the ability to acquire and process rock samples. In the near future, the Mars Sample Return (MSR) mission, a cooperative project of NASA and the European Space Agency, will likely use a lightweight rover to drive out and collect samples and bring them back to an Earth return vehicle. This rover will use an unprecedented level of autonomy because of the limited lifetime of a return rocket on the Martian surface and the desire to obtain samples from distant crater walls.

This paper considers the problem of learning to recognize different terrains from color imagery in a fully automatic fashion, using the robot's mechanical sensors as supervision.We present a probabilistic framework in which the visual information and the mechanical supervision interact to learn the available terrain types.Within this framework, a novel supervised dimensionality reduction method is proposed, in which the automatic supervision provided by the robot helps select better lower dimensional representations, more suitable for the discrimination task at hand.Incorporating supervision into the dimensionality reduction process is important, as some terrains might be visually similar but induce very different robot mobility.Therefore, choosing a lower dimensional visual representation adequately is expected to improve the vision-based terrain learning and the final classification performance.This is the first work that proposes automatically supervised dimensionality reduction in a probabilistic framework using the supervision coming from the robot's sensors.The proposed method stands in between methods for reasoning under uncertainty using probabilistic models and methods for learning the underlying structure of the data.The proposed approach has been tested on field test data collected by an autonomous robot while driving on soil, gravel and asphalt.Although the supervision might be ambiguous or noisy, our experiments show that it helps build a more appropriate lower dimensional visual representation and achieves improved terrain recognition performance compared to unsupervised learning methods.

In this paper, an algorithm for autonomous stair climbing with a tracked vehicle is presented. The proposed method achieves robust performance under real-world conditions, without assuming prior knowledge of the stair geometry, the dynamics of the vehicle's interaction with the stair surface, or lighting conditions. The approach relies on fast and accurate estimation of the robot's heading and its position relative to the stair boundaries. An extended Kalman filter is used for quaternion-based attitude estimation, fusing rotational velocity measurements from a 3-axial gyroscope, and measurements of the stair edges acquired with an onboard camera. A two-tiered controller, comprised of a centering- and a heading-control module, utilizes the estimates to guide the robot rapidly, safely, and accurately upstairs. Both the theoretical analysis and implementation of the algorithm are presented in detail, and extensive experimental results demonstrating the algorithm's performance are described.

This paper presents an approach for slip prediction from a distance for wheeled ground robots using visual information as input. Large amounts of slippage which can occur on certain surfaces, such as sandy slopes, will negatively affect rover mobility. Therefore, obtaining information about slip before entering such terrain can be very useful for better planning and avoiding these areas. To address this problem, terrain appearance and geometry information about map cells are correlated to the slip measured by the rover while traversing each cell. This relationship is learned from previous experience, so slip can be predicted remotely from visual information only. The proposed method consists of terrain type recognition and nonlinear regression modeling. The method has been implemented and tested offline on several off-road terrains including: soil, sand, gravel, and woodchips. The final slip prediction error is about 20%. The system is intended for improved navigation on steep slopes and rough terrain for Mars rovers. © 2006 Wiley Periodicals, Inc.