Depth Completion from Sparse 3D Maps for Automated Defect Quantification

For lifeline systems, routine visual inspections play a vital role in gathering structure-level data to inform asset maintenance strategies. Current research has shown that data repeatability and objectivity may be enhanced with computer vision and robotics, whereby manual collection, cataloguing, and quantification of defects is automated. For instance, low-cost lidar sensors integrated with a robotic platform can collect 3D geometric information of a structure via simultaneous localization and mapping (SLAM), with image data fusion allowing for defect measurement. While low-cost lidars provide a fast and inexpensive way of obtaining defect measurements, maps they produce often lack the density required for accurate quantification. To remedy this, we combine depth completion enhanced point cloud data from a lidar with labelled image data to extract a more complete surface estimate in physical scale. Given a 3D map, image data with known camera poses, and camera intrinsic calibrations, our approach first transforms the 3D map to a depth map (within camera frame) through ray casting. Depth maps are then densified using a depth completion algorithm that employs image processing techniques tailored to suit sparse lidar map data. Lastly, the combination of labeled defect images and dense depth maps is exploited to extract high-resolution area measurements for defects such as spalls and delaminations. The accuracy of our method has been assessed by comparing results to those obtained from non-depth completed data and to ground truth measurements obtained using a high-resolution monocular camera with manual scale input to each image. Using a data set containing six concrete area defects, our method was shown to reduce error, on average, by 14.2%, with depth completed results having an average area error of 5.3% from the ground truth. Our results show that depth completion on sparse 3D maps can be effective in fine-scale defect quantification, providing a more complete assessment of lifeline system condition using affordable sensors.