Planetary remote sensing (RS) missions are returning an ever increasing volume of data from across the Solar System. This wealth of data, much of it with a high spatial resolution, presents major challenges as well as opportunities. It is becoming increasingly difficult to fully interrogate large RS datasets such as the High Resolution Imaging Science Experiment (HiRISE) images of Mars (McEwen et al. 2010). The time required to survey all relevant images at full resolution can be daunting for all but the largest teams.Advances in machine learning provide a way to overcome these challenges, by automating the initial surveying of planetary RS data, and providing a more accessible dataset, which highlights textural features of interest to the project.The Novelty or Anomaly Hunter – HiRISE employs a deep learning convolutional neural network (DNN) to classify HiRISE images (LeCun et al. 2015; Simonyan and Zisserman 2015; He et al. 2016). A set of ontological classes was designed, which covered the complete range of textures at the site. These consisted of broad “terrain types” rather than formal geomorphological units. The aim of the project was to classify the Exo-Mars Rosalind Franklin Rover (Vago et al. 2017) landing site, and identify features such as aeolian bedforms or blockfields which might present localised hazards to rover operations (e.g. Rothrock et al. 2016). This would provide a useful input to traversability analysis. The focus was thus on detection, rather than digitisation. Producing a formal geomorphological map was beyond the scope of the project.Four broad categories of classes were selected; non-bedrock surfaces, bedrock surfaces, aeolian bedforms, and boulder fields.The surface classes were subdivided in terms of their metre scale relief and apparent roughness upon visual inspection. Bedrock classes exhibited clearly defined texture and relief, suggestive of outcrops, while non-bedrock was interpreted to consist of regolith or loose materials. Both were further subdivided according to the degree to texture present.The bedform classes were distinguished from the non bedrock surfaces by the presence of clear aeolian ripple forms (Balme et al. 2008; Balme et al. 2017). They were subdivided based upon both the scale of the bedforms, and whether they were continuous or discontinuous. Large isolated ripples were labelled individually, but this was not practical for large fields of smaller discontinuous ripples. These were thus classified based on whether they overlaid bedrock, or non bedrock surfaces. A very small number of sites within the study area also exhibit rectilinear ripples. These were only found on a large scale.Finally boulder patches consist of block fields, and regions of boulder strewn ground. Individual blocks were too small to label, so patches of boulder covered ground were classified.These classes were used to manually label a set of ~1500 training images, each being a small 128-128m “framelet” extracted from the larger HiRISE image. From these examples, the DNN learned to classify the entire site according to the prescribed classification scheme. The model output consisted of a classified raster image, of the same dimensions as the original HiRISE image. This was colour coded and overlain on the HiRISE images for further analysis.NOAH-H performed very well when identifying the very distinct classes such as bedforms, boulder fields and areas of fractured ground. Distinguishing between surface classes proved less reliable. This is likely due to the fact that many of these classes form a continuous variation, and so cannot be divided into discrete types with 100% reliability. When similar classes are grouped, and all bedrock or non bedrock terrains are considered together, the reliability of the model increased dramatically.The majority of confusion occurred within these broader groups, rather than between them. The final run of the model produced a mean Intersection over Union (IoU) of 74.15% for the full class list and 92.33% for the grouped classes.A set of sample locations were also studied to determine how the geomorphology was represented in the output data. This analysis broadly supported the results of the IoU analysis. The pixel scale results were not always found to be a perfect match, due to subtle variations within and between classes. The model sometimes struggled with “fuzzy” boundaries between regions of contrasting terrains.However, it was found that even in cases where some individual pixels were misclassified, the classification of the area as a whole was frequently still both useful and reliable. While only 53% of sampled locations were found to be correctly classified at the pixel scale, 72% were correct when the landscape of the area as a whole was considered. When classes were combined into groups, this increased to 88% of the sampled locations.The model results are thus most useful when considered at the “landscape scale”. It provides a very reliable guide to the distribution of terrains within an area and will provide a valuable tool for geomorphological study. It is already being applied to detection of aeolian hazards at the Oxia Planum site (See: EPSC2020-572).ReferencesBalme M, Berman DC, Bourke MC, Zimbelman JR (2008) Transverse Aeolian Ridges (TARs) on Mars. Geomorphology 101:703–720. doi: 10.1016/j.geomorph.2008.03.011Balme M, Robson E, Barnes R, et al. (2017) Surface-based 3D measurements of small aeolian bedforms on Mars and implications for estimating ExoMars rover traversability hazards. Planet Space Sci 153:39–53. doi: 10.1016/j.pss.2017.12.008He K, Zhang X, Ren S, Sun J (2016) Deep Residual Learning for Image Recognition. Proc. IEEE Conf. Comput. Vis. Pattern Recognition,. pp 770–778LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. doi: 10.1038/nature14539McEwen AS, Banks ME, Baugh N, et al. (2010) The High Resolution Imaging Science Experiment (HiRISE) during MRO’s Primary Science Phase (PSP). Icarus 205:2–37. doi: 10.1016/j.icarus.2009.04.023Rothrock B, Kennedy R, Cunningham C, et al. (2016) SPOC: Deep Learning-based Terrain Classification for Mars Rover Missions. AIAA Sp 2016. doi: 10.2514/6.2016-5539Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc. pp 1–14Vago JL, Westall F, Coates AJ, et al. (2017) Habitability on Early Mars and the Search for Biosignatures with the ExoMars Rover. Astrobiology 17:471–510. doi: 10.1089/ast.2016.1533

A deep learning (DL) terrain classification system, the Novelty and Anomaly Hunter - HiRISE (NOAH-H) was used to produce a terrain map of Mawrth Vallis, Mars. With it, we digitised the extent and distribution of transverse aeolian ridges (TARs), a common type of martian aeolian bedform. We present maps of the site, classifying terrain into descriptive classes and interpretive groups. TAR density maps are calculated, and the network output is compared to a manually produced map of TAR density, highlighting the differences in approach and results between these methods. Even when mapping on a small scale, humans must divide the terrain into coherent patches in order to map a large area in a reasonable time frame. Conversely, the speed of DL systems enables mapping on the pixel scale, producing a more detailed product, but one which is also "noisier", and less immediately informative. There are pros and cons to both approaches. A morphological map of Marth Vallis, Mars, has been created, classifying variations in surface texture into 14 descriptive classes.A deep learning (DL) convolutional neural network was trained to predict these classes in further HiRISE images, which had not been used for training.The resulting classified rasters were orthorectified and mosaicked using ArcGIS.Appropriate classes from the resulting map were compared with manual digitisation of the spatial densities of Transverse Aeolian Ridges (TARs).This comparison highlights the different scales at which human and DL mapping takes place, and that the two datasets have different strengths and weaknesses.The speed at which the network can complete its task allows it to attempt a higher level of fidelity than is possible for a human.Derived maps of the density of boulders and TARs were also produced using both the DL and manual datasets.

We present a map of Oxia Planum, Mars, the landing site for the ExoMars Rover. This shows surface texture and aeolian bedform distribution, classified using a deep learning (DL) system. A hierarchical classification scheme was developed, categorising the surface textures observed at the site. This was then used to train a DL network, the ‘Novelty or Anomaly Hunter – HiRISE’ (NOAH-H). The DL applied the classification scheme across a wider area than could have been mapped manually. The result showed strong agreement with human-mapped areas reserved for validation. The resulting product is presented in two ways, representing the two principle levels of the classification scheme. ‘Descriptive classes’ are purely textural in nature, making them compatible with a machine learning approach. These are then combined into ‘interpretive groups’, broader thematic classes, which provide an interpretation of the landscape. This step allows for a more intuitive analysis of the results by human users.

We applied a deep learning terrain classification system, the ‘Novelty or Anomaly Hunter – HiRISE’ (NOAH-H), originally developed for the ExoMars landing sites in Oxia Planum and Mawrth Vallis, to the Mars 2020 Perseverance rover landing site in Jezero crater. NOAH-H successfully classified the terrain in four HiRISE images of Jezero even though the landforms in the Jezero study area were slightly different from those in the training dataset. We mosaicked the NOAH-H classified rasters and compared them with a manually generated photogeological map, and with Perseverance rover and Ingenuity helicopter images. We find that grouped NOAH-H classes correspond well with the humanmade map and that individual classes are corroborated by the available ground-truth images. We conclude that our NOAH-H products can be refined for feeding into traversability analysis of the ExoMars Rosalind Franklin rover landing site at Oxia Planum and that they can also be used to aid the photogeological mapping process.

Wheeled robots typically depend on a mix of Wheel Odometry (WO) and Visual Odometry (VO) for localization. This includes inference of the slip of the wheels, representing different soil interactions. This paper investigates how slip predictions derived from forward vision or drive current, for example, can be fused with WO and VO. Our solution is based on the Extended Kalman Filter (EKF), using either point slip measurements, a slip prediction model, or a slip profile as a state. The investigation also compares different VO measurement periods, studying how VO effort trades with accuracy for each slip prediction scheme. The solutions all provide improved localization accuracy and hint at the intriguing possibility of slip-based SLAM.

A new strategy is proposed for scheduling Visual Odometry (VO) measurements for wheeled ground vehicles. Rather than having a fixed interval or distance between image acquisitions, we propose to trigger VO based on covariances from an Adaptive Extended Kalman Filter. The adopted model uses process noise to drive wheel slip estimation, which, when correctly identified, can be used with Wheel Odometry to provide frequent position estimates. When more dynamic terrain is detected, more VO measurements are scheduled to maintain localization accuracy. On the other hand, when the terrain is stable, VO usage is limited. The system is validated in a simple one-dimensional case using data captured during field trials using a representative rover. The results are promising as trajectories that were subjected to large errors are corrected.

View Video Presentation: https://doi.org/10.2514/6.2021-1757.vid A new method is proposed to estimate robotic rover's position corrected with slip. Our solution integrates the slip estimation into an Extended Kalman Filter fusing Wheel Odometry (WO) and Visual Odometry (VO). The approach can handle correlation between the slip and the rover's position estimation, and occasional errors in any of VO or WO measurements. Furthermore, it is possible to tune the model to put more emphasis on WO (e.g. when no slip is expected and thus reduce the number of VO measurements) or to rely more on VO (high slippage variability). Accurate tracking of uncertainty offers a route to adaptive use of VO, saving energy when conditions permit. The proposed model is validated in a simple one-dimensional case using data captured during field trials on a representative rover. Results are promising as the position estimation is consistent even for various VO update periods. The model is also compared with other sensor fusion algorithms. Finally, we provide an example of how a failure in VO measurement is dealt with by the proposed solution.

A new approach is presented to estimate the motion of a mobile platform using a fusion of visual odometry (VO) and wheel odometry (WO). An Extended Kalman Filter is used with an augmented state vector including the wheel slip. The new approach allows for flexible use of VO, which is central to slip estimation but consumes significant electrical power. The approach can be used in the future to adaptively optimise the number of VO measurements. Keywords—mobile platform, Visual Odometry, wheel slip, Extended Kalman Filter

Manual classification of large-scale orbital images of Mars by geologists is essential to support rover landing site selection, and eventually, operations. Extensive classification is, however, often prohibitive given the scale of the data and limited availability of specialist experts. To address this challenge, we developed a Deep Learning based system called NO-AH-H (Novelty or Anomaly Hunter - HiRISE) to automate this process. NOAH-H can provide pixel level, annotated classifications of terrain seen in HiRISE images, based on a 14 class ontology defined by experts in geomorphology. The system prototype was developed and evaluated during the recent Exo-Mars site selection process with encouraging results. This paper reports on the NOAH-H study findings.

In this investigation a deep learning terrain classification system, the "Novelty or Anomaly Hunter - HiRISE" (NOAH-H), was used to classify High Resolution Imaging Science Experiment (HiRISE) images of Oxia Planum and Mawrth Vallis. A set of ontological classes was developed that covered the variety of surface textures and aeolian bedforms present at both sites. Labelled type-examples of these classes were used to train a Deep Neural Network (DNN) to perform semantic segmentation in order to identify these classes in further HiRISE images. This contribution discusses the methods and results of the study from a geomorphologists perspective, providing a case study applying machine learning to a landscape classification task. Our aim is to highlight considerations about how to compile training datasets, select ontological classes, and understand what such systems can and cannot do. We highlight issues that arise when adapting a traditional planetary mapping workflow to the production of training data. We discuss both the pixel scale accuracy of the model, and how qualitative factors can influence the reliability and usability of the output. We conclude that "landscape level" reliability is critical for the use of the output raster by humans. The output can often be more useful than pixel scale accuracy statistics would suggest, however the product must be treated with caution, and not considered a final arbiter of geological origin. A good understanding of how and why the model classifies different landscape features is vital to interpreting it reliably. When used appropriately the classified raster provides a good indication of the prevalence and distribution of different terrain types, and informs our understanding of the study areas. We thus conclude that it is fit for purpose, and suitable for use in further work.

The observation of landforms, outcrops and small features within a (Martian) landscape is key to the understanding of its geologic past as well as present environmental conditions. Studies of such features have – for example – revealed the nature of streambeds at Gale Crater, and allowed to study Aeolian bedforms as they were encountered by the Curiosity, and Spirit rovers. With two active rovers (Opportunity, Curiosity) currently on Mars, and two more to be launched in 2020 (ExoMars, Mars2020), the imaging data sets are a huge, growing resource, which need to be explored as best as possible. LabelMars (www.labelmars.net) is a citizen science activity to collect geological annotations of Martian rover navigation camera images. As part of the ESA NOAH (Novelty Or Anomaly Hunter) project it will provide a large, high quality dataset to develop stateof-the-art machine vision algorithms for autonomous science detection, targeted at future rover missions.

The European Robotic Goal-Oriented Autonomous Controller ERGO (http://www.h2020-ergo.eu/) is one of the six space robotic projects in the frame of the PERASPERA SRC (http://www.h2020-peraspera.eu/). Its goal is to provide an Autonomy Framework capable of operating at different levels of autonomy, from tele-operations to full on-board autonomy. Even though it has been originally conceived for space robotics, its domain independent design facilitates its application to any terrestrial robotic system. This paper presents the approach followed, current status and future steps.

Beyond the current ExoMars programme, the European Space Agency (ESA) is investigating a range of technology developments and exploration mission opportunities leading to a future Mars Sample Return Mission (MSR), a critical next step in the exploration of Mars. To fulfil their scientific objectives, all of these missions require an arm with a long reach capable of performing a variety of tasks in stringent environmental conditions, such as low gravity sampling and precise sample handling and insertion. As part of an activity co-funded by the UK Space Agency, a consortium of UK companies has developed LARAD, a Lightweight Advanced Robotic Arm Demonstrator to address some of the underlying challenges related to both the design as well as operation of long arms. This paper describes the current state-of-the-art in planetary robotics and provides an overview of the top-level architecture, mechanical and mechanism design, electrical and software architectures for the LARAD demonstrator.

The SAFER project (Sample Acquisition Field Experiment with a Rover), led by RAL Space and sponsored by the European Space Agency, was successfully concluded by a Mars Mission Simulation campaign in the Atacama Desert in October 2013. The objectives of SAFER were manifolds: Firstly, bring three ExoMars instruments prototypes together on a rover platform. Secondly, perform field trials in a Mars analogue in order to acquire valuable insight on the strategies for approaching a science target, investigating it, and sampling it to investigate possible traces of life. Lastly, the SAFER project targeted to implement a highly realistic scenario with a dual team, one with the rover system in a Mars analogue location, while all navigation and science investigation decisions would be taken remotely by a separate team, fully isolated from the field. The Atacama Desert in Chile was selected during the study as suitable Mars analogue for SAFER: Its extent, variety of landscapes, high quality Mars analogue and geological characteristics are all of high relevance for a Mars mission simulation. SAFER used an early version of ESA’s 2018 ExoMars rover, provided by Astrium, fitted with a trio of ExoMars prototype instruments, including the AUPE-2 PANCAM prototype from Aberystwyth University, the WISDOM ground penetrating radar prototype from LATMOS, and a CLUPI prototype provided by Space-X. All three instruments were used to identify promising sites for subsurface excavation. During the Mars mission simulation a field trials team composed of instruments engineers, geologists, and field trials engineers was on location in the Atacama Desert in Chile. The remote control team was in parallel overseeing the rover operations from the Satellite Applications Catapult facility in Harwell, UK. The remote control team was composed of instrument operators, key members of the ExoMars programme team, ExoMars instruments principal investigators and a geologist. The controllers used a video wall to combine data from the rover’s instruments with their own 3D planning maps to help the remote team to analyse instruments data, elaborate a science strategy, and produce plans of activities for each sol, as it would be performed during a Mars exploration mission. Each plan was dispatched to the local team in the field who then uploaded it to the rover, while trying to remain as ‘invisible’ as possible for the remote operators. The SAFER campaign was a success and its results are opening the way to successful, safe, and optimized planetary exploration for the European Space Agency: Firstly, remote operations with an average of two Martian sols per day were achieved, including the acquisition of valuable instruments data. Drilling down to more than a meter was achieved by manual operation, and the samples obtained are being documented to help cross calibrating the ground penetrating instrument data and support future Mars exploration missions. Secondly, SAFER unique outcomes and lessons learnt will be transmitted to the European Space Agency’s planetary exploration teams; a key outcome has been the importance of well-defined interfaces between the rover and instruments, the importance of the remote operations team composition, and valuable operations lessons learnt demonstrating feasible activities to be achieved in a single Martian sol. Valuable instruments operations feedback was also acquired and will be presented in the final paper. Finally, SAFER opens the way to efficient, easy, and cost effective future field trials activities; an extensive expertise in the organization of such trials has been developed, as well as a detailed knowledge of the Atacama region which has proven to be a highly representative analogue for Mars mission simulations.

Under the umbrella of the European Space Agency (ESA) StarTiger program, a rapid prototyping study called Seeker was initiated. A range of partners from space and nonspace sectors were brought together to develop a prototype Mars rover system capable of autonomously exploring several kilometers of highly representative Mars terrain over a three‐day period. This paper reports on our approach and the final field trials that took place in the Atacama Desert, Chile. Long‐range navigation and the associated remote rover field trials are a new departure for ESA, and this activity therefore represents a novel initiative in this area. The primary focus was to determine if current computer vision and artificial intelligence based software could enable such a capability on Mars, given the current limit of around 200 m per Martian day. The paper does not seek to introduce new theoretical techniques or compare various approaches, but it offers a unique perspective on their behavior in a highly representative environment. The final system autonomously navigated 5.05 km in highly representative terrain during one day. This work is part of a wider effort to achieve a step change in autonomous capability for future Mars/lunar exploration rover platforms.

This paper reports on select operations and autonomy aspects of a recent ESA study called SAFER. The study sought to investigate operations strategies for a surface mission such as ExoMars Rover. In order to help prepare for ESA’s first mobile robotic mission on Mars SAFER investigated aspects of the proposed operations strategy in a representative environment. An early ExoMars Rover chassis prototype equipped with several ExoMars payload instrument breadboards was located in the Atacama Desert in Chile with a remote operations and science team situated in the UK to simulate operations over a one week period. The rover system was equipped with full on-board navigation autonomy and partially automated instrument operations. Through-out the course of the week the science and operations team explored a pre-selected region of interest using the rover and payload instruments. This paper presents the results of the field trial from an operations tools perspective and comments on the role of autonomy in such a mission.

The development of complex autonomous surface robotic systems will be critical to future planetary exploration missions. These missions will encounter dynamically changing environments with high levels of uncertainty and will require the use of specialist test facilities to fully test these systems. This paper presents the results of a study conducted on behalf of the European Space Agency (ESA) by a consortium composed of RAL Space, Astrium and SCISYS. It investigated the feasibility of a European facility for the validation and verification (V&V) of landed robotic and autonomous systems aimed primarily at the exploration of extra-terrestrial planetary surfaces. This facility would focus its activities on three key aspects: the support of rigorous Field Trials to gather “ground truth” datasets, the simulation and modelling of robotic and autonomous systems including the critical correlation with the field test data, and the fostering of a European network of facilities and expertise to support these activities. By branching out beyond the space sector, it is anticipated that it will also be able to support a range of similar terrestrial applications.