Deep learning-based classification and instance segmentation of leakage-area and scaling images of shield tunnel linings

This paper presents an approach for the integrated process of classification and instance segmentation of leakage-area and scaling images from shield tunnel linings. For this purpose, the previously established dataset of leakage-area images by the authors is enlarged by means of adding scaling ones. Afterwards, data augmentation is implemented to enrich the database in the classification dataset, and the augmented classification dataset contains 5776 images. The instance segmentation dataset is subsequently enlarged through original images without any data augmentation, including 1496 images. Then a residual net with 101 layers (i.e., ResNet-101) is applied to the classification dataset to obtain a model that can identify leakage-area and scaling images from those of shield tunnel linings. The ResNet-101 classification model achieves an accuracy of 93.37% in terms of testing classification dataset. Moreover, a mask region-based convolutional neural network (Mask R-CNN) is utilized to perform instance segmentation of leakage areas and scaling in the images classified by the ResNet-101 model. The segmentation results of the Mask R-CNN model show 96.1% and 95.6% average precision (AP) with intersection over union (IoU) of 0.5 for bounding box and mask predication, respectively. By using the proposed approach, the leakage-area and scaling defects can be automatically classified and quantified with an overall accuracy of 89.3%, which is quite promising compared to the inherent uncertainty in geotechnical engineering. Image mosaicing is finally applied to provide inspectors with the intuitive observation of the location and distribution information of defects on the tunnel lining.