Disparity Weighted Loss For Semantic Segmentation Of Driving Scenes

Convolutional neural networks are the state of the art methods for semantic segmentation but their resource consumption hinders their usability for real-time mobile robotics applications. Recent works have focused on designing lightweight networks that require less resources, but their efficiency is accompanied with a drop in performance. In this work, we propose a pixel-wise weighting of the cross-entropy loss with the disparity map in order to give more importance to close objects during the optimisation procedure of the network. This weighting is applied to two lightweight networks, with different efficiency/performance trade-offs, that were designed for real-time autonomous driving. These networks are trained on CamVid and Cityscapes datasets and the disparity maps are obtained with an off-the-shelf unsupervised depth estimation network. Our method does not increase the number of parameters of the network nor imply any further manual labeling. This weighting is evaluated on both the regular mean intersection over union (mIoU) and a close-range mIoU. Compared to the standard weighting scheme, this new loss weighting improves the mIoU and the IoU of pertinent classes for autonomous driving especially at close range.