Multi-resolution Cascaded Network with Depth-similar Residual Module for Real-time Semantic Segmentation on RGB-D Images

Multi-class indoor semantic segmentation using deep fully convolutional neural networks on RGB images has been widely used in scene parsing and human-computer interaction. Due to the wide application of depth information sensors, we can get more understanding of geographic location information from the depth information channel, but it also leads to high computational cost and memory usage. In this paper, we present a real-time deep neural network for semantic segmentation tasks on RGB-D images. First, we use an intuitive and efficient convolution operation to approximate the depth information to the pixel operation without adding additional parameters, which can be easily integrated into the deep convolutional neural network. Then, we use a multi-resolution branching structure and train the network with appropriate label guidance as the loss function to obtain a high-quality performance of semantic segmentation. The proposed approach demonstrates real-time inference on datasets NYUv2 and SUN RGB-D with a good balance of accuracy and speed on a single GPU card.