Object Detection in 3D Coral Ecosystem Maps from Multiple Image Sequences.

Coral reefs are biologically diverse and structurally complex ecosystems that have been severely affected by natural and anthropogenic stressors. Consequently, there is a need for rapid and accurate ecological assessment of coral reefs, but current approaches entail time-consuming manual data acquisition and analysis. We propose a scheme to identify and localize individual entities within the coral reef ecosystem as distinct 3D objects and assess its performance. Given 2D region proposals in an RGB image, our method generates, for each 2D region proposal, a 3D region proposal based on an existing annotated 3D reef reconstruction and the intrinsic and extrinsic camera parameters associated with the RGB image. The annotated 3D reef reconstruction is generated using a commercial Structure-from-Motion (SfM) software and a previously designed multiview convolutional neural network (CNN) for 3D semantic segmentation. As individual coral reef entities are often viewed in multiple images, a 3D bounding-box merging strategy coupled with an overlap criterion are used to combine multiple 3D region proposals into a single 3D object prediction for the purpose of classification and localization. Experimental results and comparison with the Frustum PointNet architecture show the efficacy of the proposed scheme on coral reef survey images.