1 Introduction & Scope3D vision (mapping, localization, navigation, science target recognition etc.) using Planetary Rover imaging requires high-level test assets, including a “ground truth” against which the processing results (rover locations, 3D maps) can be compared. Whilst end-to-end simulation for functional testing is realized by visualization of a drone-based DTM (Digital Terrain Model), the accuracy and robustness of vision-based navigation and 3D mapping can only be verified by high-fidelity data sets. The approach followed in the EU Horizon-2020 Project ADE [3] used a terrestrial-captured image data set for high- and medium resolution (2mm / 3dm grid size) DTM generation of a representative Mars-analog environment, followed by batch rendering to be presented to the respective 3D vision components (Visual Odometry – VO, and stereovision-based point cloud generation):Capturing terrestrial & drone-based images for photogrammetric reconstruction using ground control points (GCPs) COTS (Commercial-Off-The-Shelf) compilation of 3D textured models in different resolutions using Structure-from-Motion (SfM) Fusion of the gained textured point clouds in the visualization component PRo3D, and batch rendering of simulated stereo images at poses along a Rover trajectory Using these images to validate / evaluate vision-based navigation and mapping frameworks. 2 Ground Truth 3D Data AssemblySeveral images (6000x4000 pixels, Figure 1 bottom-left) on a test site at the Canary Islands were taken with a SONY ILCE-6500 digital camera. For DTM geo-referencing, 4 points were dGPS-measured (Figure 1, top). The reconstruction tool used was CapturingReality, using the following workflow:Import (a total of 1754) images and Ground Control Points (GCPs). Geo-registration: Identified GCPs on the images. Image alignment/registration. Find matching points, calculate camera poses and internal geometry, and a sparse 3D point cloud (Figure 1, bottom-right). Model generation for a complete dense 3D mesh in high quality. Texture mapping on the dense 3D model. Ortho image and DSM computation (49948 x 30768 pixels) with spatial resolution of 2mm in epsg:32628-WGS 84/UTM zone 28N coordinate system. A similar technique was used for an image sequence captured by a drone flown by ESA in medium height, gaining a DTM with 3cm resolution. Without GCPs, a co-registration was possible within the following visualization component (see next section). See the various 3D data sets on Figure 2.3 Simulation with PRo3DPRo3D [1] allows planetary scientists fluent navigation through a detailed geospatial environment with a visual experience close to field work. It offers much of the required functionality to generate large volumes of reference images with a fidelity close to field captures and a clear relation to the geometry of the terrain and the capturing position. The fidelity is achieved by a high resolution of the DTMs and its image textures. Combination of textured point clouds (OPCs – Ordered Point Clouds) in various resolutions is easily possible by superimposing very high resolution data sets onto global medium / low resolution data sets (Figure 3). PRo3D’s batch rendering enables mass-production of such ground truth data by defining many different viewpoints from which to render different types of images, including camera animations between such viewpoints.4 Testing the approach using a customer applicationThe demonstrated approach allows to verify navigation and reconstruction algorithms with simulated imagery. The DTMs and navigation poses resulting from processing this imagery can be directly and accurately compared with the original DTM & image poses from which the images were rendered. Thus, deviations between the ground truth and the resulting reconstruction as well as image poses can be more efficiently investigated.To test an application, 101 stereo image pairs along a straight 10m path were rendered and fed into ORB-SLAM [6] (https://github.com/Phylliida/orbslam-windows) VO with satisfactory results (Figure 4).5 ConclusionWe present a high-fidelity method to generate realistic rendered ground truth images for validating planetary 3D vision mechanisms (VO / stereo mapping / SLAM), based on terrestrial capturing of planetary analog environment by means of the PRo3D viewer’s batch render capability. The approach is currently in use for preparation of ExoMars PanCam 3D Vision, and HERA 3D Vision [4], [5].Future work can include albedo maps in connection with a shader and artificial illumination, the usage of models that support ambiguities on Cartesian DTMs (e.g. overhangs, caves), adding different types of noise to simulate real conditions, and the augmentation of the landscapes by additional (real captured) rocks [2]. On the application side, adding noise to image content and poses, longer trajectories, multiple locations for mapping, and the combination / data fusion with other simulated sensors (LIDAR, INS) can be envisaged. References[1] Barnes R., Gupta S., Traxler C., Hesina G., Ortner T., Paar G., Huber B., Juhart K., Fritz L., Nauschnegg B., Muller J.P., Tao Y. and Bauer A. Geological analysis of Martian rover-derived Digital Outcrop Models using the 3D visualisation tool, Planetary Robotics 3D Viewer - PRo3D. In Planetary Mapping: Methods, Tools for Scientific Analysis and Exploration, Volume 5, Issue 7, pp 285-307, July 2018.[2] Paar, G., Traxler, C., Sidla, O., Bechtold, A., Koeberl, C. “Science Autonomy Training by Visual Simulation”. International Symposium on Artificial Intelligence, Robotics and Automation in Space (I-SAIRAS), 2020.[3] Ocón, J., et al. "ADE: Autonomous DEcision making in very long traverses." International Symposium on Artificial Intelligence, Robotics and Automation in Space (I-SAIRAS), 2020.[4] Maria del Pilar Caballo Perucha, Rebecca Nowak, et. Al. Didymos images´ simulation for 3D reconstruction within HERA project. Abstract and oral presentation at Europlanet Science Congress 2020. Virtual Meeting. 21st September – 9th October 2020. Space Missions to Small Bodies: Planetary Defense[5] Maria del Pilar Caballo Perucha, Rebecca Nowak, et. Al. Imaging Simulations of HERA Didymos Approach. Mission & Campaign Design. Abstract and E-poster at 7th IAA Planetary Defense Conference. Online Event. 26th- 30th April 2021.[6] Mur-Artal, Raul, and Juan D. Tardós. "Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras." IEEE Transactions on Robotics5 (2017): 1255-1262.AcknowledgmentsWe thank the European Commission and the members of the PERASPERA programme support activity for their support and guidance. This work receives funding from the European Union’s Horizon 2020 Research and Innovation programme under Grant Agreement No 821988. We thank the ESA Robotics Section / Martin Azkarate for providing the aerial DTM.

Modern and ancient fluvial-deltaic systems on Earth contain highly diverse ecosystems in all terrestrial climates. Fluvial and lacustrine deposits have been discovered on Mars by the NASA Mars Science Laboratory rover Curiosity, and may be present in Oxia Planum, where the ESA/ROSCOSMOS ExoMars rover Rosalind Franklin is set to land in 2023. The primary aim of the ExoMars mission is to search for signs of past and present life on Mars. Whilst fluvio-deltaic-lacustrine sandstones and mudstones are high priority targets for sampling and drilling, it is important to obtain information on the palaeoenvironmental context of these deposits during mission exploration. The geometries of sedimentary structures and distribution of sedimentary facies within fluvial deposits provide information which can be used to reconstruct the geometries and flow parameters of these ancient systems. This provides us with quantitative means with which to make inferences on the ancient climate of Mars, and aids decision making with regards to rover science operations. Here we present a detailed quantitative 3D analysis of fluvial sedimentary architecture on Mars using rover image data. We used the 3D visualization software tool PRo3D1 to render the Shaler outcrop, observed at Yellowknife Bay by the NASA Mars Science Laboratory Rover, Curiosity2, as a scaled 3D textured model using the PRoViP 3D vision processing software3, and to map out key sedimentological features in order to characterize their geometry and dimensions, following existing facies descriptions4 . Mastcam data taken from different rover locations was processed into 3D surfaces and spatially matched to Navcam stereo-panoramas to create a digital outcrop model (DOM). The Shaler DOM was constructed using 17 Mastcam stereo-panoramas taken on Sols 120-121 and 309-324. A 30 m x 13 m area of the NE-SW trending outcrop was analysed. Sedimentary facies, key bounding surfaces and sedimentary structures were mapped out on the DOM (Fig. 1) and the dip and strike of lithological boundaries, key bounding surfaces, and cross-laminations were measured directly from the DOM. Apparent widths and thicknesses of the layers and cross-lamination sets were measured. Regularly spaced, sedimentary logs were collected and matched to illustrate the detailed internal structures of the outcrops analysed. Four types of sedimentary structures were identified; low-angle cross strata dipping to the SE (Fig. 2), ~ 50 cm thick; convex up, sub-parallel undulating laminations forming structures with 20-40 cm amplitude and 2 m wavelength; single sets of trough cross-laminations and compound, stacked cosets of trough cross-laminations, with thicknesses on average 9 cm, and yielding a common palaeoflow direction to the NE and SW. These data allow us to reconstruct the internal architecture of a fluvial bar-form which forms the Shaler outcrop, and quantify the key geometries and their spatial relationships in three-dimensions. These data are highly useful in providing context and relative timings for environmental reconstruction.

This paper was initially intended to report on the outcome of the twice postponed demonstration mission of the ARCHES project. Due to the global COVID pandemic, it has been postponed from 2020, then 2021, to 2022. Nevertheless, the development of our concepts and integration has progressed rapidly, and some of the preliminary results are worthwhile to share with the community to drive the dialog on robotics planetary exploration strategies. This paper includes an overview of the planned 4-week campaign, as well as the vision and relevance of the mission towards the planned official space missions. Furthermore, the cooperative aspect of the robotic teams, the scientific motivation, the sub task achievements are summarised.

Balme, M. R.; Curtis-Rouse, M. C.; Banham, S.; Barnes, D.; Barnes, R.; Bauer, A.; Bedford, C.; Bridges, J.; Butcher, F. E. G.; Caballo, P.; Caldwell, A.; Coates, A.; Cousins, C.; Davis, J.; Dequaire, J.; Edwards, P.; Fawdon, P.; Furuya, K.; Gadd, M.; Get, P.; Griffiths, A.; Grindrod, P. M.; Gunn, M.; Gupta, S.; Hansen, R.; Harris, J. K.; Holt, J.; Huber, B.; Huntly, C.; Hutchinson, I.; Jackson, L.; Kay, S.; Kybert, S.; Lerman, H. N.; McHugh, M.; McMahon, W.; Muller, J.-P.; Paar, G.; Preston, L. J.; Schwenzer, S.; Stabbins, R.; Tao, Y.; Traxler, C; Turner, S.; Tyler, L.; Venn, S.; Walker, H.; Wright, J. and Yeomans, B. (2017). UK Space Agency “Mars Utah Rover Field Investigation 2016” (MURFI 2016): overview of mission, aims and progress. In: 48th Lunar and Planetary Science Conference, 20-24 Mar 2017, Houston.

Panoramic camera systems on robots exploring the surface of Mars are used to collect images of terrain and rock outcrops which they encounter along their traverse. Image mosaics from these cameras are essential in mapping the surface geology and selecting locations for analysis by other instruments on the rover's payload. 2-D images do not truly portray the depth of field of features within an image, nor their 3-D geometry. This paper describes a new 3-D visualization software tool for geological analysis of Martian rover-derived Digital Outcrop Models created using photogrammetric processing of stereo-images using the Planetary Robotics Vision Processing tool developed for 3-D vision processing of ExoMars PanCam and Mars 2020 Mastcam-Z data. Digital Outcrop Models are rendered in real time in the Planetary Robotics 3-D Viewer PRo3D, allowing scientists to roam outcrops as in a terrestrial field campaign. Digitization of point, line, and polyline features is used for measuring the physical dimensions of geological features and communicating interpretations. Dip and strike of bedding and fractures is measured by digitizing a polyline along the contact or fracture trace, through which a best fit plane is plotted. The attitude of this plane is calculated in the software. Here we apply these tools to analysis of sedimentary rock outcrops and quantification of the geometry of fracture systems encountered by the science teams of NASA's Mars Exploration Rover Opportunity and Mars Science Laboratory rover Curiosity. We show the benefits PRo3D allows for visualization and collection of geological interpretations and analyses from rover-derived stereo-images. Plain Language Summary Key data returned from robots exploring the surface of Mars are the images they take of the landscape and rock formations. These are sent back to Earth for detailed investigation and analysis by the science teams. It is difficult to collect reliable measurements from photographs, as they do not truly represent the three-dimensionality of the features within them. In this paper, we present a new 3-D visualization software tool, PRo3D, which enables visualization of 3-D digital models of rock outcrops imaged by robots exploring the surface of Mars. These 3-D models are constructed from mosaicked photographs taken by the stereo panoramic cameras which are positioned on a mast on the rover. This provides a huge advantage to scientists who want to study and analyze the terrain and geology of exposed rock outcrops which surround the rover. Here we apply the tools available in PRo3D to sedimentological and structural analysis of 3-D Digital Outcrop Models of four areas explored by the Mars Exploration Rover Opportunity and Mars Science Laboratory Curiosity rover science teams and show that this method of 3-D visualization and analysis allows scientists to carry out important procedures that would be conducted in a terrestrial field geology campaign.

The 2016 Mars Utah Rover Field Investigation (MURFI) was a Mars rover field trial run by the UK Space Agency in association with the Canadian Space Agency's 2015/2016 Mars Sample Return Analogue Deployment mission. MURFI had over 50 participants from 15 different institutions around the UK and abroad. The objectives of MURFI were to develop experience and leadership within the UK in running future rover field trials; to prepare the UK planetary community for involvement in the European Space Agency/Roscosmos ExoMars 2020 rover mission; and to assess how ExoMars operations may differ from previous rover missions. Hence, the wider MURFI trial included a ten-day (or ten-'sol') ExoMars rover-like simulation. This comprised an operations team and control centre in the UK, and a rover platform in Utah, equipped with instruments to emulate the ExoMars rovers remote sensing and analytical suite. The operations team operated in 'blind mode', where the only available data came from the rover instruments, and daily tactical planning was performed under strict time constraints to simulate real communications windows. The designated science goal of the MURFI ExoMars rover-like simulation was to locate in-situ bedrock, at a site suitable for sub-surface coresampling, in order to detect signs of ancient life. Prior to "landing", the only information available to the operations team was Mars-equivalent satellite remote sensing data, which were used for both geologic and hazard (e.g., slopes, loose soil) characterisation of the area. During each sol of the mission, the operations team sent driving instructions and imaging/analysis targeting commands, which were then enacted by the field team and rover-controllers in Utah. During the ten-sol mission, the rover drove over 100 m and obtained hundreds of images and supporting observations, allowing the operations team to build up geologic hypotheses for the local area and select possible drilling locations. On sol 9, the team obtained a subsurface core sample that was then analyzed by the Raman spectrometer. Following the conclusion of the ExoMars-like component of MURFI, the operations and field team came together to evaluate the successes and failures of the mission, and discuss lessons learnt for ExoMars rover and future field trials. Key outcomes relevant to ExoMars rover included a key recognition of the importance of field trials for (i) understanding how to operate the ExoMars rover instruments as a suite, (ii) building an operations planning team that can work well together under strict time-limited pressure, (iii) developing new processes and workflows relevant to the ExoMars rover, (iv) understanding the limits and benefits of satellite mapping and (v) practicing efficient geological interpretation of outcrops and landscapes from rover-based data, by comparing the outcomes of the simulated mission with post-trial, in-situ field observations. In addition, MURFI was perceived by all who participated as a vital learning experience, especially for early and mid-career members of the team, and also demonstrated the UK capability of implementing a large rover field trial. The lessons learnt from MURFI are therefore relevant both to ExoMars rover, and to future rover field trials.

Balme, M. R.; Curtis-Rouse, M. C.; Banham, S.; Barnes, D.; Barnes, R.; Bauer, A.; Bedford, C.; Bridges, J.; Butcher, F. E. G.; Caballo, P.; Caldwell, A.; Coates, A.; Cousins, C.; Davis, J.; Dequaire, J.; Edwards, P.; Fawdon, P.; Furuya, K.; Gadd, M.; Get, P.; Griffiths, A.; Grindrod, P. M.; Gunn, M.; Gupta, S.; Hansen, R.; Harris, J. K.; Holt, J.; Huber, B.; Huntly, C.; Hutchinson, I.; Jackson, L.; Kay, S.; Kybert, S.; Lerman, H. N.; McHugh, M.; McMahon, W.; Muller, J.-P.; Paar, G.; Preston, L. J.; Schwenzer, S.; Stabbins, R.; Tao, Y.; Traxler, C; Turner, S.; Tyler, L.; Venn, S.; Walker, H.; Wright, J. and Yeomans, B. (2017). UK Space Agency “Mars Utah Rover Field Investigation 2016” (MURFI 2016): overview of mission, aims and progress. In: 48th Lunar and Planetary Science Conference, 20-24 Mar 2017, Houston.

The Mars Utah Rover Field Investigation “MURFI 2016” is a Mars Rover field analogue mission run by the UK Space Agency (UKSA) in collaboration with the Canadian Space Agency (CSA). MURFI 2016 took place between 22nd October and 13th November 2016 and consisted of a field team including an instrumented Rover platform, at the field site near Hanksville (Utah, USA), and an ‘Operations Team’ based in the Mission Control Centre (MOC) at the Harwell Campus near Oxford in the UK. The field site was chosen based on the collaboration with the CSA and its Mars-like local geology. It was used by the CSA in 2015 for Mars Rover trials, and in 2016, several teams used the site, each with their own designated working areas. The two main aims of MURFI 2016 were (i) to develop logistical and leadership experience in running field trials within the UKSA, and (ii) to provide members of the Mars Science community with Rover Operations experience, and hence to build expertise that could be used in the 2020 ExoMars Rover mission, or other future Rover missions. Because MURFI 2016 was the first solely UKSA-led Rover analogue trial, the most important objective was to learn how to best implement Rover trials in general. This included aspects of planning, logistics, field safety, MOC setup and support, communications, person management and science team development. Some aspects were based on past experience from previous trials but the focus was on ‘learning through experience’ - especially in terms of the Operations Team, who each took on a variety of roles during the mission.

DURING THE MARS UTAH ROVER FIELD INVESTIGATION (MURFI) ANALOGUE ROVER TRIALS. R. Barnes1, S.Gupta1, M. Gunn2, G. Paar3, B. Huber3, A. Bauer3, K. Furya3, M. P. Caballo-Perucha3, C. Traxler4, G. Hesina4, T. Ortner4, J. P. Muller5, Y. Tao5, S. G. Banham1, J. Harris6, M. Balme7. 1Imperial College, London, UK, robert.barnes@imperial.ac.uk, 2Aberystwyth University, UK 3Joanneum Research, Graz, Austria 4VRVis, Vienna, Austria, 5Mullard Space Science Laboratory, University College London 6Birkbeck, University of London, UK, 7School of Physical Sciences, Open University, UK.

The 3D documentation of the tunnel surface during construction requires fast and robust measurement systems. In the solution proposed in this paper, during tunnel advance a single camera is taking pictures of the tunnel surface from several positions. The recorded images are automatically processed to gain a 3D tunnel surface model. Image acquisition is realized by the tunneling/advance/driving personnel close to the tunnel face (= the front end of the advance). Based on the following fully automatic analysis/evaluation, a decision on the quality of the outbreak can be made within a few minutes. This paper describes the image recording system and conditions as well as the stereo-photogrammetry based workflow for the continuously merged dense 3D reconstruction of the entire advance region. Geo-reference is realized by means of signalized targets that are automatically detected in the images. We report on the results of recent testing under real construction conditions, and conclude with prospects for further development in terms of on-site performance.

(1) Nottingham Geospatial Institute, University of Nottingham, Nottingham, UK, (2) Mullard Space Science Lab, University College London, Hombury St. Mary, Dorking, UK, (3) JOANNEUM RESEARCH Forschungsgesellschaft mbH, Steyrergasse 17, 8010 Graz, AUSTRIA, (4) Department for Geodesy and Geoinformation Science, Technical University of Berlin, Berlin, GERMANY, (5) VRVis Zentrum fuer Virtual Reality und Visualisierung Forschungs-GmbH, Vienna, AUSTRIA, (6) MIIGAIK Extraterrestrial Lab (MEXLAB), Moscow State University of Geodesy and Cartography (MIIGAiK), Moscow, RUSSIAN FEDERATION

Muller, Y. Tao, P. Sidiropoulos, V. Yershov, J.G. Morley, J. Sprinks, G. Paar, B. Huber A. Bauer, K. Willner, C. Traxler, Imaging Group, Mullard Space Science Laboratory, University College London, Dept. of Space & Climate Physics, Holmbury St Mary, Surrey, RH5 6NT, UK, j.muller@ucl.ac.uk; Nottingham Geospatial Institute, University of Nottingham, University Park, Nottingham, NG7 2RD, UK, jeremy.morley@nottingham.ac.uk; Joanneum Research F-GmbH, Steyrergasse 17, 8010 Graz, Austria, gerhard.paar@joanneum.at; Dept. of Geodesy & Geoinformation Science, Technical University Berlin, 10623 Berlin, Germany, konrad.willner@tu-berlin.de; VRVis Zentrum für VR und Visualisierung F-GmbH, Donau-City-Strasse, Vienna, Austria, traxler@vrvis.at

In the scope of the presented work, an Automated Terrestrial Laser Scanner (ATLS) was setup to continuously monitor potential avalanche slopes in high-alpine terrain, located in the Western Austrian Alps during two consecutive winters (2012/13 & 2013/14). To acquire and analyse the data, an elaborate scanner control and data processing framework was developed and implemented. The results show, that the ATLS setup is able to provide an almost complete series of scans of the target areas every 12h, at a mean point spacing of 0.4-1.1 m and an accuracy of ± 0.05 m (la), plus a distance dependent error of <; 20 ppm. An overview of limitations and further potential of these techniques conclude the paper.

This article discusses long-range terrestrial laser scanning (TLS) as a monitoring technique focusing on recent geomorphic changes at the near-terminus zone of the Hinteres Langtalkar rock glacier (46°59′N, 12°47′E). This rock glacier is characterised by extraordinarily high movement rates at its lower part since the mid-1990s. The surface of this collapsing part of the rock glacier is characterised by a very disturbed topography, which causes problems for terrestrial as well as remote sensing monitoring methods. The lack of textural information makes approaches such as optical flow detection in grey-scale images non-applicable at this study area. Long-range TLS is completing the dataset with distance measurements leading to 3D point clouds, which are afterwards converted into accurate 3D models. Analysis of surface kinematics derived from five digital elevation models (DEM) obtained during TLS campaigns between 2001–2006 shows promising results covering this lack of data in unfavourable terrain.

1 Computer Science Department (steve.pugh@aber.ac.uk), 2 Institute of Maths and Physics, Aberystwyth University, UK, 3 German Aerospace Centre (DLR), Institute of Planetary Research, Berlin, Germany. 4 Joanneum Research, Graz, Austria , 5 Earth & Planetary Sciences, Birkbeck College, University of London, UK, 6 Department of Physics and Astronomy, University of Leicester, Leicester, UK, 7 UCL, Mullard Space Science Laboratory, Surrey, UK.

A scientific camera system would provide valuable geological context from the surface for lunar lander missions. Here, we describe the PanCam instrument from the ESA ExoMars rover and its possible adaptation for the proposed ESA lunar lander. The scientific objectives of the ESA ExoMars rover are designed to answer several key questions in the search for life on Mars. The ExoMars PanCam instrument will set the geological and morphological context for that mission. We describe the PanCam scientific objectives in geology, and atmospheric science, and 3D vision objectives. We also describe the design of PanCam, which includes a stereo pair of Wide Angle Cameras (WACs), each of which has a filter wheel, and a High Resolution Camera for close up investigations. The cameras are housed in an optical bench (OB) and electrical interface is provided via the PanCam Interface Unit (PIU). Additional hardware items include a PanCam Calibration Target (PCT). We also briefly discuss some PanCam testing during field trials. In addition, we examine how such a ‘Lunar PanCam’ could be adapted for use on the Lunar surface on the proposed ESA lunar lander.