Planetary rovers can use onboard data analysis to adapt their measurement plan on the fly, improving the science value of data collected between commands from Earth. This paper describes the implementation of an adaptive sampling algorithm used by PIXL, the X-ray fluorescence spectrometer of the Mars 2020 Perseverance rover. PIXL is deployed using the rover arm to measure X-ray spectra of rocks with a scan density of several thousand points over an area of typically 5 x 7 mm. The adaptive sampling algorithm is programmed to recognize points of interest and to increase the signal-to-noise ratio at those locations by performing longer integrations. Two approaches are used to formulate the sampling rules based on past quantification data: 1) Expressions that isolate particular regions within a ternary compositional diagram, and 2) Machine learning rules that threshold for a high weight percent of particular compounds. The design of the rulesets are outlined and the performance of the algorithm is quantified using measurements from the surface of Mars. To our knowledge, PIXL's adaptive sampling represents the first autonomous decision-making based on real-time compositional analysis by a spacecraft on the surface of another planet.

The AEGIS (Autonomous Exploration for Gathering Increased Science) intelligent targeting software system has been in use on the Mars Science Laboratory (MSL) mission since 2016.The system allows on-board autonomous selection of targets for the ChemCam remote geochemistry instrument based on analysis of images taken by the rover.This paper describes the deployment of AEGIS to MSL and the operational use of the system since rollout to science operations in May of 2016.We describe how MSL Science Operations have adapted to include autonomous target selection, both in procedures and in exploration strategies, and how mission planners and scientists have used AEGIS autonomy to enhance their work on the MSL mission.

Limitations on interplanetary communications create operations latencies and slow progress in planetary surface missions, with particular challenges to narrow-field-of-view science instruments requiring precise targeting. The AEGIS (Autonomous Exploration for Gathering Increased Science) autonomous targeting system has been in routine use on NASA's Curiosity Mars rover since May 2016, selecting targets for the ChemCam remote geochemical spectrometer instrument. AEGIS operates in two modes; in autonomous target selection, it identifies geological targets in images from the rover's navigation cameras, choosing for itself targets that match the parameters specified by mission scientists the most, and immediately measures them with ChemCam, without Earth in the loop. In autonomous pointing refinement, the system corrects small pointing errors on the order of a few milliradians in observations targeted by operators on Earth, allowing very small features to be observed reliably on the first attempt. AEGIS consistently recognizes and selects the geological materials requested of it, parsing and interpreting geological scenes in tens to hundreds of seconds with very limited computing resources. Performance in autonomously selecting the most desired target material over the last 2.5 kilometers of driving into previously unexplored terrain exceeds 93% (where ~24% is expected without intelligent targeting), and all observations resulted in a successful geochemical observation. The system has substantially reduced lost time on the mission and markedly increased the pace of data collection with ChemCam. AEGIS autonomy has rapidly been adopted as an exploration tool by the mission scientists and has influenced their strategy for exploring the rover's environment.

To perform more complex space exploration activities with limited human intervention, an intelligent system must be able not only to sense its environment, but also to interpret the sensory data it acquires. Rock Segmentation Through Edge Regrouping is an autonomous perception algorithm for scientific analysis that is deployed on Mars. It conducts onboard analysis of images collected by the Mars Exploration Rover Opportunity and provides a list of closed rock contours to the Autonomous Exploration for Gathering Increased Science software module, which then prioritizes the identified rocks for subsequent targeting based on preferences expressed by scientists. Rock Segmentation Through Edge Regrouping processes 1 Kx1 K images in 600-900 s on the MER RAD6000 flight processor, clocked to operate at 20 million instructions per second, with a guaranteed high-water memory footprint of less than 4 megabytes of RAM. In all runs on Mars with rocks or outcrop present, the top 10 returned targets have been valid rocks or outcrop with one exception, which was a dark patch of soil. In several runs in which there were no rocks present, the algorithm correctly returned no detections. A nearly integer-only parallel version of the algorithm has been demonstrated on a Tilera TILE64 multicore processor.

research-article Share on Onboard machine learning classification of images by a cubesat in Earth orbit Authors: David R. Thompson Calif. Inst. of Tech. Calif. Inst. of Tech.View Profile , Alphan Altinok Calif. Inst. of Tech. Calif. Inst. of Tech.View Profile , Ben Bornstein Calif. Inst. of Tech. Calif. Inst. of Tech.View Profile , Steve A. Chien Calif. Inst. of Tech. Calif. Inst. of Tech.View Profile , Joshua Doubleday Calif. Inst. of Tech. Calif. Inst. of Tech.View Profile , John Bellardo California Polytechnic State Univ., San Luis Obispo California Polytechnic State Univ., San Luis ObispoView Profile , Kiri L. Wagstaff Calif. Inst. of Tech. Calif. Inst. of Tech.View Profile Authors Info & Claims AI MattersVolume 1Issue 4June 2015pp 38–40https://doi.org/10.1145/2757001.2757010Published:16 June 2015Publication History 6citation376DownloadsMetricsTotal Citations6Total Downloads376Last 12 Months23Last 6 weeks7 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Automatic cloud recognition promises significant improvements in Earth science remote sensing. At any time, more than half of Earth's surface is covered by clouds, obscuring images and atmospheric measurements. This is particularly problematic for CubeSats, a new generation of small, low‐orbiting spacecraft with very limited communications bandwidth. Such spacecraft can use image analysis to autonomously select clear scenes for prioritized downlink. More agile spacecraft can also benefit from cloud screening by retargeting observations to cloud‐free areas. This could significantly improve the science yield of instruments such as the Orbiting Carbon Observatory 3 mission. However, most existing cloud detection algorithms are not suitable for these applications, because they require calibrated and georectified spectral data, which is not typically available onboard. Here, we describe a statistical machine‐learning method for real‐time autonomous scene interpretation using a visible camera with no radiometric calibration. A random forest classifies cloud and clear pixels based on local patterns of image texture. We report on experimental evaluation of images from the International Space Station (ISS) and present results from a deployment onboard the IPEX spacecraft. This demonstrates actual execution in flight and provides some preliminary lessons learned about operational use. It is a rare example of a machine‐learning system deployed to an autonomous spacecraft. To our knowledge, it is also the first instance of significant artificial intelligence deployed on board a CubeSat and the first ever deployment of visible image‐based cloud screening onboard any operational spacecraft.

AEGIS (Autonomous Exploration for Gathering Increased Science) is a software suite that will imminently be operational aboard NASA's Curiosity Mars rover, allowing the rover to autonomously detect and prioritize targets in its surroundings, and acquire geochemical spectra using its ChemCam instrument. ChemCam, a Laser-Induced Breakdown Spectrometer (LIBS), is normally used to study targets selected by scientists using images taken by the rover on a previous sol and relayed by Mars orbiters to Earth. During certain mission phases, ground-based target selection entails significant delays and the use of limited communication bandwidth to send the images. AEGIS will allow the science team to define the properties of preferred targets, and obtain geochemical data more quickly, at lower data penalty, without the extra ground-inthe-loop step. The system uses advanced image analysis techniques to find targets in images taken by the rover's stereo navigation cameras (NavCam), and can rank, filter, and select targets based on properties selected by the science team. AEGIS can also be used to analyze images from ChemCam's Remote Micro Imager (RMI) context camera, allowing it to autonomously target very fine-scale features - such as veins in a rock outcrop - which are too small to detect with the range and resolution of NavCam. AEGIS allows science activities to be conducted in a greater range of mission conditions, and saves precious time and command cycles during the rover's surface mission. The system is currently undergoing initial tests and checkouts aboard the rover, and is expected to be operational by late 2015. Other current activities are focused on science team training and the development of target profiles for the environments in which AEGIS is expected to be used on Mars.

The Autonomous Exploration for Gathering Increased Science (AEGIS) system enables automated science data collection by a planetary rover. AEGIS analyzes rover images onboard to detect pre-defined science features of interest, enabling targeted instrument data to be acquired immediately with no delays for ground communication. AEGIS analyses images to detect candidate targets and then selects specific features for followup measurements based on scientist-specified objectives. This paper describes the application of AEGIS for use with the Mars Science Laboratory (MSL) mission ChemCam spectrometer. ChemCam uses a Laser Induced Breakdown Spectrometer (LIBS) to analyze the elemental composition of rocks and soil. AEGIS applies to ChemCam in two ways. The first involves automated targeting of ChemCam during or after long drives by finding rock targets in Navigation Camera images. The second involves refining ChemCam pointing by detecting small targets, such as veins or concretions, in Remote Micro Imager images. This paper describes both of these applications.

Rockster-MER is an autonomous perception capability that was uploaded to the Mars Exploration Rover Opportunity in December 2009. This software provides the vision front end for a larger software system known as AEGIS (Autonomous Exploration for Gathering Increased Science), which was recently named 2011 NASA Software of the Year. As the first step in AEGIS, Rockster-MER analyzes an image captured by the rover, and detects and automatically identifies the boundary contours of rocks and regions of outcrop present in the scene. This initial segmentation step reduces the data volume from millions of pixels into hundreds (or fewer) of rock contours. Subsequent stages of AEGIS then prioritize the best rocks according to scientist- defined preferences and take high-resolution, follow-up observations. Rockster-MER has performed robustly from the outset on the Mars surface under challenging conditions. Rockster-MER is a specially adapted, embedded version of the original Rockster algorithm (Rock Segmentation Through Edge Regrouping, (NPO- 44417) Software Tech Briefs, September 2008, p. 25). Although the new version performs the same basic task as the original code, the software has been (1) significantly upgraded to overcome the severe onboard re source limitations (CPU, memory, power, time) and (2) bulletproofed through code reviews and extensive testing and profiling to avoid the occurrence of faults. Because of the limited computational power of the RAD6000 flight processor on Opportunity (roughly two orders of magnitude slower than a modern workstation), the algorithm was heavily tuned to improve its speed. Several functional elements of the original algorithm were removed as a result of an extensive cost/benefit analysis conducted on a large set of archived rover images. The algorithm was also required to operate below a stringent 4MB high-water memory ceiling; hence, numerous tricks and strategies were introduced to reduce the memory footprint. Local filtering operations were re-coded to operate on horizontal data stripes across the image. Data types were reduced to smaller sizes where possible. Binary- valued intermediate results were squeezed into a more compact, one-bit-per-pixel representation through bit packing and bit manipulation macros. An estimated 16-fold reduction in memory footprint relative to the original Rockster algorithm was achieved. The resulting memory footprint is less than four times the base image size. Also, memory allocation calls were modified to draw from a static pool and consolidated to reduce memory management overhead and fragmentation. Rockster-MER has now been run onboard Opportunity numerous times as part of AEGIS with exceptional performance. Sample results are available on the AEGIS website at http://aegis.jpl.nasa.gov.

The TextureCam project is developing a 'smart camera' that can classify geologic surfaces in planetary images. This would allow autonomous spacecraft to collect data opportunisitcally during intervals between communications with Earth, such as during long traverses. Its surface classifications can identify new targets that were not anticipated in advance. The spacecraft might use this information to target these features with high-resolution instruments such as spectrometers nd narrow-field cameras. Classifications could also inform data 'triage' decisions, identifying high value images for prioritized downlink. Finally, the surface classification can serve as compressed maps of image content. Each of these strategies can improve the science data returned at each command cycle and speed reconnaissance during site survey and astrobiology investigation. Our first year of development has completed the image analysis algorithms and validated them in software tests. Here we survey these initial results and explore several application areas relevant to Mars and beyond.

Imaging spectrometers are valuable instruments for space exploration, but their large data volumes limit the number of scenes that can be downlinked. Missions could improve science yield by acquiring surplus images and analyzing them onboard the spacecraft. This onboard analysis could generate surficial maps, summarizing scenes in a bandwidth-efficient manner to indicate data cubes that warrant a complete downlink. Additionally, onboard analysis could detect targets of opportunity and trigger immediate automated follow-up measurements by the spacecraft. Here, we report a first step toward these goals with demonstrations of fully automatic hyperspectral scene analysis, feature discovery, and mapping onboard the Earth Observing One (EO-1) spacecraft. We describe a series of overflights in which the spacecraft analyzes a scene and produces summary maps along with lists of salient features for prioritized downlink. The onboard system uses a superpixel endmember detection approach to identify compositionally distinctive features in each image. This procedure suits the limited computing resources of the EO-1 flight processor. It requires very little advance information about the anticipated spectral features, but the resulting surface composition maps agree well with canonical human interpretations. Identical spacecraft commands detect outlier spectral features in multiple scenarios having different constituents and imaging conditions.

We report on trace gas and major atmospheric constituents results obtained by the Vehicle Cabin Atmosphere Monitor (VCAM) following almost two years of operation aboard the International Space Station (ISS). VCAM is an autonomous environmental monitor based on a highly compact gas chromatograph/quadrupole ion trap mass spectrometer. It was flown to the International Space Station (ISS) on shuttle mission STS-131 and commenced operations on June 2010. VCAM is capable of providing measurements of both parts-per-billion (ppb) levels of volatile trace-gas constituents, and of the atmospheric major constituents (nitrogen, oxygen, argon, and carbon dioxide) in a space vehicle or station. It is designed to operate autonomously and maintenance-free, approximately once per day, with a self-contained gas supply sufficient for a one-year lifetime. VCAM’s performance is sufficient to detect and identify 90% of the target compounds at their 180-day Spacecraft Maximum Allowable Concentration levels.

We present a demonstration of onboard hyperspectral image processing with the potential to reduce mission downlink requirements. The system detects spectral endmembers and then uses them to map units of surface material. This summarizes the content of the scene, reveals spectral anomalies warranting fast response, and reduces data volume by two orders of magnitude. We have integrated this system into the Autonomous Science craft Experiment for operational use onboard the Earth Observing One (EO-1) Spacecraft. The system does not require prior knowledge about spectra of interest. We report on a series of trial overflights in which identical spacecraft commands are effective for autonomous spectral discovery and mapping for varied target features, scenes and imaging conditions.