System, methods, and other embodiments described herein relate to improving calibration of an onboard sensor of a vehicle. In one embodiment, a method includes, in response to acquiring sensor data from a surrounding environment of the vehicle using the onboard sensor, analyzing the sensor data to determine calibration parameters for the onboard sensor. The method includes identifying a suitability parameter that characterizes how well the surrounding environment provides for determining the calibration parameters. The method includes generating annotations within a map that specify at least the suitability parameter for a location associated with the sensor data. In further aspects, the method includes identifying, from the map, a calibration route for the vehicle that is a deviation from a current route in response to determining that the calibration state of the onboard sensor does not satisfy the calibration threshold.

Lane-level maps provide crucial detail to autonomous vehicle perception and decision making systems. Many common HD maps require human annotators to painstakingly label attributes such as lane geometry, connectivity, and speed limit. In this paper, we propose a method to derive these attributes automatically. Our method uses lane features and previous vehicle experience to generate potentials in a rectified raster map. We then infer the driving lanes within a Markov random field. Moreover, the lane topologies are extracted and represented by parsing the roadway into different segments based on the number of driving lanes and generating lane transitions between adjacent road segments. Lane attributes, such as speed limit and stop locations, are inferred from statistical analysis of previous vehicle experiences. Our approach is supported by experiments demonstrating effective automatic lane map generation on real-world roads.

A system and method for calibrating sensors may include one or more processors, a first sensor configured to obtain a two-dimensional image, a second sensor configured to obtain three-dimensional point cloud data, and a memory device. The memory device stores a data collection module and a calibration module. The data collection module has instructions that configure the one or more processors to obtain the two-dimensional image and the three-dimensional point cloud data. The calibration module has instructions that configure the one or more processors to determine and project a three-dimensional point cloud edge of the three-dimensional point cloud data onto the two-dimensional image edge, apply a branch-and-bound optimization algorithm to a plurality of rigid body transforms, determine a lowest cost transform of the plurality of rigid body transforms using the branch-and-bound optimization algorithm, and calibrate the first sensor with the second sensor using the lowest cost transform.

Systems and methods for vehicular navigation are disclosed herein. One embodiment receives, from one or more sensors, sensor data pertaining to a roadway section that is proximate to a vehicle; generates segmented sensor data to identify, in the roadway section, one or more boundary lines of one or more lanes; determines, from the sensor data, a direction of travel associated with at least one of the one or more lanes; applies a graphical model to the segmented sensor data to generate an output that includes a set of discrete points corresponding to the one or more boundary lines; generates an objective map of the roadway section from the set of discrete points; and uses the objective map to assist the vehicle in navigating the roadway section.

System, methods, and other embodiments described herein relate to predicting a presence of occluded objects from a robotic device. In one embodiment, a method includes, in response to acquiring sensor data about a surrounding environment, analyzing the sensor data to identify a perceived object in the surrounding environment by determining at least a class of the perceived object. The method includes determining a presence factor associated with the perceived object according to an observation model. The presence factor indicates a likelihood of an occluded object existing in an occluded region associated with the perceived object. The method includes controlling one or more systems of the robotic device according to the presence factor.

System, methods, and other embodiments described herein relate to calibrating a light detection and ranging (LiDAR) sensor with a camera sensor. In one embodiment, a method includes controlling i) the LiDAR sensor to acquire point cloud data, and ii) the camera sensor to acquire an image. The point cloud data and the image at least partially overlap in relation to a field of view of a surrounding environment. The method includes projecting the point cloud data into the image to form a combined image. The method includes adjusting sensor parameters of the LiDAR sensor and the camera sensor according to the combined image to calibrate the LiDAR sensor and the camera sensor together.

A point cloud management system provides labels for each point within a point cloud map. The point cloud management system also provides a method to localize a vehicle using the labeled point cloud. The point cloud management system identifies objects within a scene using an obtained image. The point cloud management system labels the identified objects to register the identified objects against the point cloud. The registration of the objects is then used to localize the vehicle.

The systems and methods described herein disclose detecting events in a vehicular environment using vehicle behavior. As described here, vehicles, either manual or autonomous, that detect an event in the environment will operate to respond to the event. As such, those movements can be used to determine if an event has occurred, even if the event cannot be determined directly. The systems and methods can include collecting detection data about a vehicle behaviors in a vehicular environment. Event behaviors can then be selected from the vehicle behaviors. A predicted event can be formulated based on the event behaviors. The predicted event and an event location can be associated in the vehicular environment. A guidance input can then be formulated for a recipient vehicle. Finally, a recipient vehicle can be navigated using the guidance input.

A map object management system including object information for objects in the environment around a location. The objects generally relating to the roadway on which a user is travelling. The map object management system provides a user with queries regarding detected objects that fail to meet confidence thresholds regarding the identification of the object. The map object management system receives answers to the queries to crowdsource uncertain or inaccurate object identifications for use in a map service to provide accurate map information to users.

System, methods, and other embodiments described herein relate to improving mapping of a surrounding environment by a mapping vehicle. In one embodiment, a method includes identifying dynamic objects within the surrounding environment that are proximate to the mapping vehicle from sensor data of at least one sensor of the mapping vehicle. The dynamic objects are trackable objects that are moving within the surrounding environment. The method includes generating paths of the dynamic objects through the surrounding environment relative to the mapping vehicle according to separate observations of the dynamic objects embodied within the sensor data. The method includes producing a map of the surrounding environment from the paths.

Determining the rigid-body transformation be-tween 2D image data and 3D point cloud data has applications for mobile robotics including sensor calibration and localizing into a prior map. Common approaches to 2D-3D registration use least-squares solvers assuming known associations often provided by heuristic front-ends, or iterative nearest-neighbor. We present a linear line-based 2D-3D registration algorithm formulated as a mixed-integer program to simultaneously solve for the correct transformation and data association. Our formulation is explicitly formulated to handle outliers, by modeling associations as integer variables. Additionally, we can constrain the registration to SE(2) to improve runtime and accuracy. We evaluate this search over multiple real-world data sets demonstrating adaptability to scene variation.

System, methods, and other embodiments described herein relate to identifying changes between models of a locality. In one embodiment, a method includes, in response to determining that a location model is available for a present environment of a vehicle, generating a current model of the present environment using at least one sensor of the vehicle. The method also includes isolating dynamic objects in the current model as a function of the location model. The method includes providing the dynamic objects to be identified and labeled.

Sparse feature-based maps provide a compact representation of the environment that admit efficient algorithms, for example simultaneous localization and mapping. These representations typically assume a static world and therefore contain static map features. However, since the world contains dynamic elements, determining when map features no longer correspond to the environment is essential for long-term utility. This work develops a feature-based model of the environment which evolves over time through feature persistence. Moreover, we augment the state-of-the-art sparse mapping model with a correlative structure that captures spatio-temporal properties, e.g. that nearby features frequently have similar persistence. We show that such relationships, typically addressed through an ad hoc formalism focusing only on feature repeatability, are crucial to evaluate through a probabilistically principled approach. The joint posterior over feature persistence can be computed efficiently and used to improve online data association decisions for localization. The proposed algorithms are validated in numerical simulation and using publicly available data sets.

This paper reports on a fast multiresolution scan matcher for local vehicle localization of self-driving cars. State-of-theart approaches to vehicle localization rely on observing road surface reflectivity with a 3D light detection and ranging (LIDAR) scanner to achieve centimeter-level accuracy. However, these approaches can often fail when faced with adverse weather conditions that obscure the view of the road paint (e.g. puddles and snowdrifts), poor road surface texture, or when road appearance degrades over time. We present a generic probabilistic method for localizing an autonomous vehicle equipped with a three-dimensional (3D) LIDAR scanner. This proposed algorithm models the world as a mixture of several Gaussians, characterizing the z-height and reflectivity distribution of the environment-which we rasterize to facilitate fast and exact multiresolution inference. Results are shown on a collection of datasets totaling over 500 km of road data covering highway, rural, residential, and urban roadways, in which we demonstrate our method to be robust through heavy snowfall and roadway repavements.

Many autonomous systems require the ability to perceive and understand motion in a dynamic environment. We present a novel algorithm that estimates this motion from raw LIDAR data in real-time without the need for segmentation or model-based tracking. The sensor data is first used to construct an occupancy grid. The foreground is then extracted via a learned background filter. Using the filtered occupancy grid, raw scene flow between successive scans is computed. Finally, we incorporate these measurements in a filtering framework to estimate temporal scene flow. We evaluate our method on the KITTI dataset.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

This paper reports on visual obstacle detection from a monocular camera for autonomous vehicles. By leveraging a textured prior map, we propose a probabilistic formulation for finding the optimal image partition that separates obstacles from groundplane. Our key insight is the use of a prior map that enables ground appearance models conditioned on prior map texture and a probabilistic optical flow vector formulation derived from known scene structure and camera egomotion. We evaluate our methods on a challenging urban setting using data collected on our autonomous platform and we demonstrate that a notion of obstacles in the camera frame can improve visual localization quality.

L'invention concerne un appareil et un procede permettant une localisation visuelle d'un systeme de camera visuelle transmettant des donnees de camera visuelle en temps reel et une unite de traitement graphique qui recoit les donnees de camera visuelles en temps reel. L'unite de traitement graphique a acces a une base de donnees d'informations cartographiques anterieures et genere une image de synthese qui est ensuite comparee aux donnees de camera visuelles en temps reel afin de determiner des donnees de position corrigees. L'unite de traitement graphique determine une position de camera sur la base des donnees de position corrigees. Un systeme de correction permettant d'appliquer la navigation du vehicule en se basant sur la position determinee de la camera peut etre utilise dans certains modes de realisation.

This paper reports on an optical visual fiducial system developed for relative-pose estimation of two ships at sea. Visual fiducials are ubiquitous in the robotics literature, however none are specifically designed for use in outdoor lighting conditions. Blooming of the CCD causes a significant bias in the estimated pose of square tags that use the outer corners as point correspondences. In this paper, we augment existing state-of-the-art visual fiducials with a border of circles that enables high accuracy, robust pose estimation. We also present a methodology for characterizing tag measurement uncertainty on a per measurement basis. We integrate these methods into a relative ship motion estimation system and support our results using outdoor imagery and field data collected aboard the USNS John Glenn and USNS Bob Hope during skin-to-skin operations.

This paper reports on a Naval Engineering Education Center (NEEC) design-build-test project focused on the development of a fully autonomous system for landing Navy unmanned aerial vehicles (UAVs) on transiting ships at sea. Our NEEC team of engineering students researched image processing techniques, estimation frameworks, and control algorithms to collaboratively learn and train in Navy-relevant autonomy. We accomplished the autonomous landing using fiducial markers detected by a camera onboard the UAV, decomposed the resulting homography into 6-degree of freedom relative-pose information between the UAV and landing platform, performed state estimation with a delayed-state extended Kalman filter, and then executed the motion control. Results for the state estimation framework and experimental tests using a motion capture system for independent ground-truth are presented. The proposed system performed successfully on a robotic testbed consisting of a micro UAV and an unmanned ground vehicle.