Multi-head Instance Segmentation of Indoor Scenes for AR/DR Applications

In order to understand the semantics of indoor scenes, e.g. to enable Augmented Reality applications for planning and design, segmenting and classifying relevant objects such as furniture items and building elements is an important prerequisite. We propose an approach for instance segmentation based on the SOLOv2 network, called SOLOv2-MH, which combines multiple classification and segmentation heads. These heads can be trained separately on different data, using a shared backbone. We provide evaluation results for different variants of the model on the Scannet benchmark, showing that it achieves state of the art performance.