System, methods, and other embodiments described herein relate to improving calibration of an onboard sensor of a vehicle. In one embodiment, a method includes, in response to acquiring sensor data from a surrounding environment of the vehicle using the onboard sensor, analyzing the sensor data to determine calibration parameters for the onboard sensor. The method includes identifying a suitability parameter that characterizes how well the surrounding environment provides for determining the calibration parameters. The method includes generating annotations within a map that specify at least the suitability parameter for a location associated with the sensor data. In further aspects, the method includes identifying, from the map, a calibration route for the vehicle that is a deviation from a current route in response to determining that the calibration state of the onboard sensor does not satisfy the calibration threshold.

A map object management system including object information for objects in the environment around a location. The objects generally relating to the roadway on which a user is travelling. The map object management system provides a user with queries regarding detected objects that fail to meet confidence thresholds regarding the identification of the object. The map object management system receives answers to the queries to crowdsource uncertain or inaccurate object identifications for use in a map service to provide accurate map information to users.

Sparse feature-based maps provide a compact representation of the environment that admit efficient algorithms, for example simultaneous localization and mapping. These representations typically assume a static world and therefore contain static map features. However, since the world contains dynamic elements, determining when map features no longer correspond to the environment is essential for long-term utility. This work develops a feature-based model of the environment which evolves over time through feature persistence. Moreover, we augment the state-of-the-art sparse mapping model with a correlative structure that captures spatio-temporal properties, e.g. that nearby features frequently have similar persistence. We show that such relationships, typically addressed through an ad hoc formalism focusing only on feature repeatability, are crucial to evaluate through a probabilistically principled approach. The joint posterior over feature persistence can be computed efficiently and used to improve online data association decisions for localization. The proposed algorithms are validated in numerical simulation and using publicly available data sets.

This paper reports on a model-assisted bundle adjustment (BA) framework in which visually-derived features are fused with an underlying three-dimensional (3D) mesh provided a priori. By using an approach inspired by the expectation-maximization (EM) class of algorithms, we introduce a hidden binary label for each visual feature that indicates if that feature is considered part of the nominal model, or if the feature corresponds to 3D structure that is absent from the model. Therefore, in addition to improved estimates of the feature locations, we can identify visual features that correspond to foreign structure on the ship hull. We show that this framework is a special case of the Gaussian max-mixtures framework, which can be efficiently incorporated into state-of-the-art graph-based simultaneous localization and mapping (SLAM) solvers.In addition, the precision of our bundle adjustment framework allows the identification of structural deviations between 3D structure inferred from bundle-adjusted camera imagery and the prior model. These structural deviations are clustered into shapes, which allow us to fuse camera-derived structure back into the 3D mesh. This augmented model can be used within a 3D photomosaicing pipeline, providing a visually intuitive 3D reconstruction of the ship hull. We evaluate our pipeline using the Bluefin Robotics hovering autonomous underwater vehicle (HAUV) surveying the SS Curtiss, where a 3D mesh derived from computer aided design (CAD) drawings serves as the prior model. In addition to more consistent visual reconstructions, we can update the prior mesh with 3D information corresponding to underwater structure, such as biofouling or manually-placed cylindrical shapes with known dimensions. (C) 2016 Elsevier B.V. All rights reserved.

This chapter reviews the concept of pose-graphPose graph simultaneous localizationLocalization and mapping (SLAM) for underwater navigationNavigation . We show that pose-graph SLAM is a generalized framework that can be applied to many diverse underwater navigation problems in marine roboticsMarine robotics . We highlight three specific examples as applied in the areas of autonomous ship hull inspectionShip hull inspection and multi-vehicle cooperative navigationMulti vehicle coordination .

Documenting underwater archaeological sites is an extremely challenging problem. Sites covering large areas are particularly daunting for traditional techniques. In this paper, we present a novel approach to this problem using both an autonomous underwater vehicle (AUV) and a diver-controlled stereo imaging platform to document the submerged BronzeAge city at Pavlopetri, Greece. The result is a three-dimensional (3D) reconstruction covering 26,600 m2 at a resolution of 2 mm/pixel, the largest-scale underwater optical 3D map, at such a resolution, in the world to date. We discuss the advances necessary to achieve this result, including i) an approach to color correct large numbers of images at varying altitudes and over varying bottom types; ii) a large-scale bundle adjustment framework that is capable of handling upward of 400,000 stereo images; and iii) a novel approach to the registration and rapid documentation of an underwater excavations area that can quickly produce maps of site change. We present visual and quantitative comparisons to the authors' previous underwater mapping approaches. (C) 2016 Wiley Periodicals, Inc.

Feature learning has become popular in robotics due to recent advances in machine learning. In this paper, we propose a novel method to utilize the high-dimensional features from these techniques as observations in Bayesian estimation problems in a real-time manner. We develop an approach that: 1) pre-processes the observations and maps them into a new space with both reduced dimensions and a linear relationship to the estimation states; and 2) estimates the uncertainty of resulting outputs using data perturbation. The result is that deep learning approaches can be combined with more traditional filtering approaches like the Kalman filter (KF) to achieve state-of-the-art real-time performance. We validate the method by presenting the first real-time application of underwater robot localization using an imaging sonar. The proposed technique shows similar localization accuracy to benchmark approaches while simultaneously achieving real-time performance.

This paper reports on a model-assisted bundle adjustment framework in which visually-derived features are fused with an underlying three-dimensional (3D) mesh provided a priori. By using an approach inspired by the expectation-maximization (EM) class of algorithms, we introduce a hidden binary label for each visual feature that indicates if that feature is considered part of the nominal model, or if the feature corresponds to 3D structure that is absent from this model. Therefore, in addition to improved estimates of the feature locations, we can also label the features based on their deviation from the model. We show that this method is a special case of the Gaussian max-mixtures framework, which can be efficiently incorporated into state-of-the-art graph-based simultaneous localization and mapping (SLAM) solvers. We provide field tests taken from the Bluefin Robotics Hovering Autonomous Underwater Vehicle (HAUV) surveying the SS Curtiss.

This paper reports on an optical visual fiducial system developed for relative-pose estimation of two ships at sea. Visual fiducials are ubiquitous in the robotics literature, however none are specifically designed for use in outdoor lighting conditions. Blooming of the CCD causes a significant bias in the estimated pose of square tags that use the outer corners as point correspondences. In this paper, we augment existing state-of-the-art visual fiducials with a border of circles that enables high accuracy, robust pose estimation. We also present a methodology for characterizing tag measurement uncertainty on a per measurement basis. We integrate these methods into a relative ship motion estimation system and support our results using outdoor imagery and field data collected aboard the USNS John Glenn and USNS Bob Hope during skin-to-skin operations.

This paper reports on a system for an autonomous underwater vehicle to perform in situ , multiple session hull inspection using long‐term simultaneous localization and mapping (SLAM). Our method assumes very little a priori knowledge, and it does not require the aid of acoustic beacons for navigation, which is a typical mode of navigation in this type of application. Our system combines recent techniques in underwater saliency‐informed visual SLAM and a method for representing the ship hull surface as a collection of many locally planar surface features. This methodology produces accurate maps that can be constructed in real‐time on consumer‐grade computing hardware. A single‐session SLAM result is initially used as a prior map for later sessions, where the robot automatically merges the multiple surveys into a common hull‐relative reference frame. To perform the relocalization step, we use a particle filter that leverages the locally planar representation of the ship hull surface, and a fast visual descriptor matching algorithm. Finally, we apply the recently developed graph sparsification tool, generic linear constraints, as a way to manage the computational complexity of the SLAM system as the robot accumulates information across multiple sessions. We show results for 20 SLAM sessions for two large vessels over the course of days, months, and even up to three years, with a total path length of approximately 10.2 km.

This paper reports on an algorithm enabling an autonomous underwater vehicle (AUV) to localize into a 3D computer aided design (CAD) model of a ship hull in situ using an optical camera and Doppler velocity log (DVL). The precision of our localization algorithm allows the identification of structural deviations between 3D structure inferred from bundle-adjusted camera imagery and the CAD model. These structural deviations are clustered into shapes, which allow us to fuse camera-derived structure into a CAD-derived 3D mesh. This augmented CAD model can be used within a 3D photomosaicing pipeline, providing a visually intuitive 3D reconstruction of the ship hull. We evaluate our algorithm on the Bluefin Robotics Hovering Autonomous Underwater Vehicle (HAUV) surveying the SS Curtiss, and provide a 3D reconstruction that fuses the CAD mesh with 3D information corresponding to underwater structure, such as biofouling.

This paper reports on a 3D photomosaicing pipeline using data collected from an autonomous underwater vehicle performing simultaneous localization and mapping (SLAM). The pipeline projects and blends 2D imaging sonar data onto a large-scale 3D mesh that is either given a priori or derived from SLAM. Compared to other methods that generate a 2D-only mosaic, our approach produces 3D models that are more structurally representative of the environment being surveyed. Additionally, our system leverages recent work in underwater SLAM using sparse point clouds derived from Doppler velocity log range returns to relax the need for a prior model. We show that the method produces reasonably accurate surface reconstruction and blending consistency, with and without the use of a prior mesh. We experimentally evaluate our approach with a Hovering Autonomous Underwater Vehicle (HAUV) performing inspection of a large underwater ship hull.

This paper reports on a factor graph simultaneous localization and mapping framework for autonomous underwater vehicle localization based on terrain-aided navigation. The method requires no prior bathymetric map and only assumes that the autonomous underwater vehicle has the ability to sparsely sense the local water column depth, such as with a bottom-looking Doppler velocity log. Since dead-reckoned navigation is accurate in short time windows, the vehicle accumulates several water column depth point clouds- or submaps-during the course of its survey. We propose an xy-alignment procedure between these submaps in order to enforce consistent bathymetric structure over time, and therefore attempt to bound long-term navigation drift. We evaluate the submap alignment method in simulation and present performance results from multiple autonomous underwater vehicle field trials.

This paper reports on a method for an autonomous underwater vehicle to perform real-time visual simultaneous localization and mapping (SLAM) on large ship hulls over multiple sessions. Along with a monocular camera, our method uses a piecewise-planar model to explicitly optimize the ship hull surface in our factor-graph framework, and anchor nodes to co-register multiple surveys. To enable realtime performance for long-term SLAM, we use the recent Generic Linear Constraints (GLC) framework to sparsify our factor-graph. This paper analyzes how our single-session SLAM techniques can be used in the GLC framework, and describes a particle filter reacquisition algorithm so that an underwater session can be automatically re-localized to a previously built SLAM graph. We provide real-world experimental results involving automated ship hull inspection, and show that our localization filter out-performs Fast Appearance-Based Mapping (FAB-MAP), a popular place-recognition system. Using our approach, we can automatically align surveys that were taken days, months, and even years apart.

This paper reports on the use of planar patches as features in a real-time simultaneous localization and mapping (SLAM) system to model smooth surfaces as piecewise-planar. This approach works well for using observed point clouds to correct odometry error, even when the point cloud is sparse. Such sparse point clouds are easily derived by Doppler velocity log sensors for underwater navigation. Each planar patch contained in this point cloud can be constrained in a factor-graph-based approach to SLAM so that neighboring patches are sufficiently coplanar so as to constrain the robot trajectory, but not so much so that the curvature of the surface is lost in the representation. To validate our approach, we simulated a virtual 6-degree of freedom robot performing a spiral-like survey of a sphere, and provide real-world experimental results for an autonomous underwater vehicle used for automated ship hull inspection. We demonstrate that using the sparse 3D point cloud greatly improves the self-consistency of the map. Furthermore, the use of our piecewise-planar framework provides an additional constraint to multi-session underwater SLAM, improving performance over monocular camera measurements alone.

This paper reports on methods for incorporating camera calibration uncertainty into a two-view sparse bundle adjustment (SBA) framework. The co-registration of two images is useful in mobile robotics for determining motion over time. These camera measurements can constrain a robot's relative poses so that the trajectory and map can be estimated in a technique known as simultaneous localization and mapping (SLAM). Here, we comment on the importance of propagating uncertainty in both feature extraction and camera calibration in visual pose-graph SLAM. We derive an improved pose covariance estimate that leverages the Unscented Transform, and compare its performance to previous methods in both simulated and experimental trials. The two experiments reported here involve data from a camera mounted on a KUKA robotic arm (where a precise ground-truth trajectory is available) and a Hovering Autonomous Underwater Vehicle (HAUV) for large-scale autonomous ship hull inspection.