Seafloor mapping is crucial for ocean exploration. Current techniques mostly involve either long-range acoustic sensors or short-range passive optical systems. This paper presents an innovative underwater flash lidar system that bridges the gap by enabling high-resolution 3D mapping at intermediate ranges. To ensure accurate registration of lidar measurements, we propose an automatic pipeline based on the Iterative Closest Point (ICP) algorithm. Preliminary field experiments demonstrate the system's potential for applications in geosciences, marine biology, archaeology and infrastructure monitoring, highlighting under-water lidar as a promising tool for detailed and efficient seafloor mapping.

Imagery has become one of the main data sources for investigating seascape spatial patterns. This is particularly true in deep-sea environments, which are only accessible with underwater vehicles. On the one hand, using collaborative web-based tools and machine learning algorithms, biological and geological features can now be massively annotated on 2D images with the support of experts. On the other hand, geomorphometrics such as slope or rugosity derived from 3D models built with structure from motion (sfm) methodology can then be used to answer spatial distribution questions. However, precise georeferencing of 2D annotations on 3D models has proven challenging for deep-sea images, due to a large mismatch between navigation obtained from underwater vehicles and the reprojected navigation computed in the process of building 3D models. In addition, although 3D models can be directly annotated, the process becomes challenging due to the low resolution of textures and the large size of the models. In this article, we propose a streamlined, open-access processing pipeline to reproject 2D image annotations onto 3D models using ray tracing. Using four underwater image datasets, we assessed the accuracy of annotation reprojection on 3D models and achieved successful georeferencing to centimetric accuracy. The combination of photogrammetric 3D models and accurate 2D annotations would allow the construction of a 3D representation of the landscape and could provide new insights into understanding species microdistribution and biotic interactions.

We set out to characterize the fine-scale processes acting on interannual dynamics of deep-sea vent fauna by using a novel approach involving a 5-yr time series of 3D photogrammetry models acquired at the Eiffel Tower sulfide edifice (Lucky Strike vent field, Mid-Atlantic Ridge). Consistently, with the overall stability of the vent edifice, total mussel cover did not undergo drastic changes, suggesting that they have been at a climax stage for at least 25 yr based on previous data. Successional patterns showed consistency over time, illustrating the dynamic equilibrium of the ecological system. In contrast, microbial mats significantly declined, possibly due to magmatic events. The remaining environmental variability consisted of decimeter-scale displacement of vent outflows, resulting from their opening or closure or from the progressive accretion of sulfide material. As a result, vent mussels showed submeter variability in the immediate vicinity of vent exits, possibly by repositioning in response to that fine-scale regime of change. As former studies were not able to quantify processes at submeter scales in complex settings, this pioneering work demonstrates the potential of 3D photogrammetry models for conducting long-term monitoring in the deep sea. We observed that the ability of mussels to displace may enable them to cope with changing local conditions in a stable system. However, the long-term stability of mussel assemblages questions their capacity to withstand large-scale disturbances and may imply a low resilience of these "climax" communities. This suggests that they may be particularly vulnerable to the negative effects of mining activities in hydrothermal ecosystems.

Imaging is increasingly used to capture information on the marine environment thanks to the improvements in imaging equipment, devices for carrying cameras and data storage in recent years. In that context, biologists, geologists, computer specialists and end-users must gather to discuss the methods and procedures for optimising the quality and quantity of data collected from images. The 4th Marine Imaging Workshop was organised from 3-6 October 2022 in Brest (France) in a hybrid mode. More than a hundred participants were welcomed in person and about 80 people attended the online sessions. The workshop was organised in a single plenary session of presentations followed by discussion sessions. These were based on dynamic polls and open questions that allowed recording of the imaging community’s current and future ideas. In addition, a whole day was dedicated to practical sessions on image analysis, data standardisation and communication tools. The format of this edition allowed the participation of a wider community, including lower-income countries, early career scientists, all working on laboratory, benthic and pelagic imaging. This article summarises the topics addressed during the workshop, particularly the outcomes of the discussion sessions for future reference and to make the workshop results available to the open public.

Underwater images are altered by the physical characteristics of the medium through which light rays pass before reaching the optical sensor. Scattering and wavelength-dependent absorption significantly modify the captured colors depending on the distance of observed elements to the image plane. In this paper, we aim to recover an image of the scene as if the water had no effect on light propagation. We introduce SUCRe, a novel method that exploits the scene’s 3D structure for underwater color restoration. By following points in multiple images and tracking their intensities at different distances to the sensor, we constrain the optimization of the parameters in an underwater image formation model and retrieve unattenuated pixel intensities. We conduct extensive quantitative and qualitative analyses of our approach in a variety of scenarios ranging from natural light to deep-sea environments using three underwater datasets acquired from real-world scenarios and one synthetic dataset. We also compare the performance of the proposed approach with that of a wide range of existing state-of-the-art methods. The results demonstrate a consistent benefit of exploiting multiple views across a spectrum of objective metrics. Our code is publicly available at github.com/clementinboittiaux/sucre.

Visual localization plays an important role in the positioning and navigation of robotics systems within previously visited environments. When visits occur over long periods of time, changes in the environment related to seasons or day-night cycles present a major challenge. Under water, the sources of variability are due to other factors such as water conditions or growth of marine organisms. Yet, it remains a major obstacle and a much less studied one, partly due to the lack of data. This paper presents a new deep-sea dataset to benchmark underwater long-term visual localization. The dataset is composed of images from four visits to the same hydrothermal vent edifice over the course of 5 years. Camera poses and a common geometry of the scene were estimated using navigation data and Structure-from-Motion. This serves as a reference when evaluating visual localization techniques. An analysis of the data provides insights about the major changes observed throughout the years. Furthermore, several well-established visual localization methods are evaluated on the dataset, showing there is still room for improvement in underwater long-term visual localization. The data is made publicly available at seanoe.org/data/00810/92226/.

This paper presents a combination of real-time and offline 3D reconstruction methods for remotely operated vehicles (ROVs) equipped with cameras used in underwater inspection and survey tasks. The real-time component is based on a stereo visual simultaneous localization and mapping algorithm and a truncated signed distance field representation for producing a coarse 3D reconstruction online. The offline component uses structure-from-motion techniques to create a dense point cloud representation of the scene which is then meshed and textured to produce a high-quality textured 3D mesh. The paper highlights the feasibility of using ROVs for vision-based 3D reconstruction in real-world scenarios and the potential of combining real-time and offline processing in practice for a range of underwater applications.

The new 6000m depth capable AUV uly x is scheduled to enter regular operational service within the French Oceanographic Fleet in 2024. Over the course of the past two years the AUV has featured a comprehensive technical validation program at sea. This article provides a description of the main features of the new system, focusing on the technological and innovative aspects introduced by the development project. An account of the main achievements of the first 76 dives of the AUV provides the outline of the path to full operational deployment.

The seismic hazard posed by submarine faults and the capacity of submarine earthquakes to trigger mass wasting are poorly understood because we lack detailed characterizations of coseismic ruptures at the seafloor. Here, we present comprehensive mapping of a seafloor rupture caused by the 2004 M w 6.3 Les Saintes earthquake on the Roseau normal fault in the Lesser Antilles. We report the visual characteristics, displacement profile, and note pronounced asymmetry of the rupture that bears similarities with well-studied subaerial normal fault ruptures. We also identify footwall-derived mass wasted debris that locally cover the coseismic rupture, and show that ground accelerations of 0.1–0.2 g can trigger submarine mass wasting events in well consolidated bedrock along unstable, over-steepened, scarps. Our study demonstrates the potential of underwater vehicles for detailed mapping of seafloor ruptures and hints at a key role for earthquakes in shaping submarine bedrock landscapes by triggering mass wasting events.

This paper addresses the lack of "push-button" software for optical marine imaging, which currently limits the use of photogrammetric approaches by a wider community. It presents and reviews an open source software, Matisse, for creating textured 3D models of complex underwater scenes from video or still images. This software, developed for non-experts, enables routine and efficient processing of underwater images into 3D models that facilitate the exploitation and analysis of underwater imagery. When vehicle navigation data are available, Matisse allows for seamless integration of such data to produce 3D reconstructions that are georeferenced and properly scaled. The software includes pre-processing tools to extract images from videos and to make corrections for color and uneven lighting. Four datasets of different 3D scenes are provided for demonstration. They include both input images and navigation and associated 3D models generated with Matisse. The datasets, captured under different survey geometries, lead to 3D models of different sizes and demonstrate the capabilities of the software. The software suite also includes a 3D scene analysis tool, 3DMetrics, which can be used to visualize 3D scenes, incorporate elevation terrain models (e.g., from high-resolution bathymetry data) and manage, extract, and export quantitative measurements for the 3D data analysis. Both software packages are publicly available.

Some recent visual-based relocalization algorithms rely on deep learning methods to perform camera pose regression from image data. This letter focuses on the loss functions that embed the error between two poses to perform deep learning based camera pose regression. Existing loss functions are either difficult-to-tune multi-objective functions or present unstable reprojection errors that rely on ground truth 3D scene points and require a two-step training. To deal with these issues, we introduce a novel loss function which is based on a multiplane homography integration. This new function does not require prior initialization and only depends on physically interpretable hyperparameters. Furthermore, the experiments carried out on well established relocalization datasets show that it minimizes best the mean square reprojection error during training when compared with existing loss functions.

The Lacaze-Duthiers Canyon is located in the western Mediterranean Sea and is long known for hosting cold-water coral colonies in the canyon head region at depths ranging from 250 to 550 m. In 2019 during the CALADU cruise, three kinds of 3D-reconstructions were applied to better understand the distribution of coral colonies, their habitat and their skeleton morphologies. The canyon's flanks were mapped using a hull-mounted echosounder and an ROV multibeam echosounder. Digital terrain models were built with resolutions of 5 and 1 m and examined in three dimensions. ROV bathymetric data collected on the canyon's flanks made it possible to highlight a series of sub-parallel structures identified as lithified sedimentary strata along which coral colonies grow. Coral assemblages were explored at four locations and photographic images were assembled using structure from motion techniques to build photogrammetric models. Coral assemblages reconstructed in 3D enabled geo-localizing and recreating coral colonies on 16 models over a total area of 4370 m2. Two colonial species, Madrepora oculata and Desmophyllum pertusum were plotted and reported on bathymetric models to interpret their location at the scale of the canyon. The coordinates and depth of the colonies were used to calculate the vertical distribution (limited to our small bathymetric exploration, between 339 and 214 m depth) and density of populations (up to 4.3 colonies per m2). The spatial coverage of the 16 assemblages measured between 100 and 600 m2 each. The sizes of the colonies were measured to analyze the population structures of both species (mean sizes of 28 cm for D. pertusum and 18 cm for M. oculata, maximum sizes 1 m and 0.5 m, respectively, bushes 2.5 m long). In addition, lost fishing gears were quantified, longlines measured and their densities calculated (0.16 m/m2, up to 0.30 m/m2). An area with exuberant orange colonies of D. pertusum was discovered for the first time in the Lacaze-Duthiers Canyon. Five deep-sea scleractinian species were collected and micro-tomographic scans computed to view their intrinsic skeleton organization. Micro-CT scans of M. oculata, D. pertusum, Desmophyllum dianthus, Caryophyllia smithii, and Dendrophyllia cornigera enabled longitudinal and transversal cuts, highlighting morphological criteria for species identification and the multidirectional examination of specimens. We observed a thin canal connecting calices along the axis of D. pertusum colonies, and separate calices along the axis of M. oculata colonies.

Hyperspectral (HS) imaging produces an image of an object across a large range of the visible spectrum, and not just the primary colors (R, G, B) of conventional cameras. It can provide valuable information for object detection, analysis of materials and processes in environmental science in the deep-sea, especially for the study of benthic environments and pollution monitoring. In this paper, we address the problem of camera calibration towards 3D hyperspectral mapping where GPS is not available, and the platform navigational sensors are not accurate enough to allow direct georeferencing of linear sensors, as is the case with traditional aerial platform methods. Our approach presents a preliminary method for 3D hyperspectral mapping that uses only image processing techniques to reduce reliance on GPS or navigation sensors. The method is based on the use of standard RGB camera coupled with the hyperspectral pushbroom camera. The main contribution is the implementation and preliminary testing of a method to relate the two cameras using image information alone. The experiments presented in this paper analyze the estimation of relative orientation and time synchronization parameters for both cameras through experiments based on epipolar geometry and Monte-Carlo simulation. All methods are designed to work with real world data.

Hyperspectral imaging has been increasingly used for underwater survey applications over the past years. As many hyperspectral cameras work as push-broom scanners, their use is usually limited to the creation of photo-mosaics based on a flat surface approximation and by interpolating the camera pose from dead-reckoning navigation. Yet, because of drift in the navigation and the mostly wrong flat surface assumption, the quality of the obtained photo-mosaics is often too low to support adequate analysis. In this paper we present an initial method for creating hyper-spectral 3D reconstructions of underwater environments. By fusing the data gathered by a classical RGB camera, an inertial navigation system and a hyperspectral push- broom camera, we show that the proposed method creates highly accurate 3D reconstructions with hyperspectral textures. We propose to combine techniques from simultaneous localization and mapping, structure-from-motion and 3D reconstruction and advantageously use them to create 3D models with hyperspectral texture, allowing us to overcome the flat surface assumption and the classical limitation of dead-reckoning navigation.

During the ODEMAR 2013 and SUBSAINTES 2017 cruises we mapped the full extent of the seafloor rupture associated with the 2004 Mw 6.3 Les Saintes extensional earthquake. Near-bottom bathymetry acquired both with ROVs and AUVs along the Roseau Fault reveal a normal fault scarp developing in an extensional graben within the Caribbean volcanic arc, between the islands of Guadeloupe and Dominica. Optical inspection during ROV dives along the scarp’s base, where fault mirrors are well-preserved, allowed us to identify and characterize the coseismic fault rupture, and measure the coseismic displacements using both laser calipers and measurements performed on video-derived, textured 3D models, with accuracies better than 1 cm. The 2004 rupture extends ~20 km along the Roseau Fault, with a vertical displacement exceeding 2.5 m at its center, and tapering towards its ends. Local variations in apparent fault slip within a single 3D model (fault lengths of ~10 to 300 m) document local deposition of gravity debris cones at the base of the scarp, extending laterally between a few to tens of m, and covering the coseismic markers. Gullies eroding the footwall and depositing debris cones on the hanging wall do not show any significant displacement. Fault scarps on either side of the gully mouth instead record significant displacements, suggesting that either erosion or deposition along the gully bottom efficiently obliterated markers of coseismic deformation. We inspected all overlapping seafloor imagery acquired in December 2013 and April 2017, >10 years after the 2004 Les Saintes earthquake, extending laterally over >3 km of the Roseau Fault rupture. Neither the bed of gullies crossing the rupture, nor the debris and rubble at the base of the fault scarp show any noticeable seafloor change indicating mass wasting and transport, and only changes in mobile sediment (e.g., ripples) can be detected between both image sets. We identified a single area, ~2m wide, with apparent deposition of pebbles during these 3.25 years period, and associated with a local mass-wasting event. These observations point towards a systematic triggering of mass-wasting during seismic events, with deposition of rubble and rocks both at dejection cones at the mouth of gullies, or at the base of fault scarp sections displaying fault mirrors, covering or obliterating the coseismic markers. Therefore, long-term erosion and deposition processes here are gravity-driven and triggered by the history and magnitude of seismic events. Similar seismic controls may enable denudation of exposed oceanic lithosphere at fault scarps developing along and flanking mid-ocean ridges.

This paper addresses the benefits and challenges of mixed reality (MR) for the exploration of deep-sea environments with remotely operated vehicles. The approach is twofold: virtual reality (VR) let the scientist explore the environment via a visual 3D model, overcoming limitations of local perception. Augmented reality (AR) concepts are designed in order to improve environment perception and interaction. The key to such concepts is the implementation of 3D visual geo-referenced terrain models from the imaging feedback gathered by the vehicle exploring its unknown surroundings. Image processing, underwater vehicle navigation, and user-friendly displays for robotic intervention are addressed in an integrated concept. A broad development programme carried out at the French Institute for Ocean Science, IFREMER, is described and illustrates technical topics and use cases. 3D perception derived from camera vision is shown to enable AR concepts that will significantly improve remote exploration and intervention in unknown natural environments. Cumulative geo-referenced 3D model building is in the process of being taken to reliable functioning in real-world underwater applications, accomplishing a milestone change in the capacity to view and understand the obscure and inaccessible deep-sea world.

The deep sea is characterized by a wide range of landscapes, including complex features where topography and currents interact to form highly heterogeneous habitats. In addition to a complex topography, hydrothermal vent environments are characterized by strong environmental gradients that structure the spatial distribution of biological communities. The role of vent fluid temperature and chemical composition on species distribution is now well understood, but investigations on the effects of the complex sulfide edifice topography are scarce. Here, we used a novel approach combining 3D photogrammetric reconstruction, in situ environmental measurements and modeling to characterize assemblage distribution on the active edifice Eiffel Tower (Lucky Strike, Mid-Atlantic Ridge). Through the analysis of a high-resolution 3D model of the edifice, we show that assemblage distribution along with hydrothermal activity vary with their position on the edifice. Although physical terrain variables had a minor effect on assemblage distribution, the distance from fluid exits explained the distribution of most assemblages. However, these particular variables did not significantly explain the distribution of medium-sized Bathymodiolus azoricus mussels, the dominant assemblage on the edifice. Similarly, proximity to fluid exits only partially accounted for the distribution of microbial mats throughout the edifice. By modeling the current-driven dispersion of hydrothermal plumes around the edifice, we demonstrated that differences in mussel sizes may be due to differences in exposure time to currents bringing plume material. For the first time, we provide evidence that hydrothermal plumes can affect faunal assemblages meters away from fluid exits and that this relatively long-distance effect of vent plumes can fully account for microbial mat distribution throughout the edifice. Our findings extend the area of influence of hydrothermal plumes on vent communities considerably beyond previous estimations and suggest that the interactions between bottom currents, topography and smoker locations should be further investigated and considered as important structuring factors at vents. This novel approach, allowing to cover large areas of the seafloor, is particularly well suited for deep environments where topography and currents interact to form complex oceanographic patterns (e.g. canyons, seamounts). Its application to larger areas and various ecosystems can significantly enhance our understanding of benthic communities’ distribution at large.

Recent advances in structure-from-motion techniques are enabling many scientific fields to benefit from the routine creation of detailed 3D models. However, for a large number of applications, only a single camera is available, due to cost or space constraints in the survey platforms. Monocular structure-from-motion raises the issue of properly estimating the scale of the 3D models, in order to later use those models for metrology. The scale can be determined from the presence of visible objects of known dimensions, or from information on the magnitude of the camera motion provided by other sensors, such as GPS. This paper addresses the problem of accurately scaling 3D models created from monocular cameras in GPS-denied environments, such as in underwater applications. Motivated by the common availability of underwater laser scalers, we present two novel approaches. A fully-calibrated method enables the use of arbitrary laser setups, while a partially-calibrated method reduces the need for calibration by only assuming parallelism on the laser beams, with no constraints on the camera. The proposed methods have several advantages with respect to the existing methods. The need for laser alignment with the optical axis of the camera is removed, together with the extremely error-prone manual identification of image points on the 3D model. The performance of the methods and their applicability was evaluated on both data generated from a realistic 3D model and data collected during an oceanographic cruise in 2017. Three separate laser configurations have been tested, encompassing nearly all possible laser setups, to evaluate the effects of terrain roughness, noise, camera perspective angle and camera-scene distance. In the real scenario, the computation of 6 independent model scale estimates using our fully-calibrated approach, produced values with standard deviation of 0.3%.