Few-shot semantic segmentation via multi-level feature extraction and multi-prototype localization

Few-shot Semantic Segmentation (FSS) segments query images only by using a few support images with ground truth. The existing methods usually extract a single prototype from the support image feature, which results in the spatial information loss. In this paper, we propose an end-to-end multi-level feature extraction and multi-prototype localization network named MMLNet for few-shot semantic segmentation, which consists of a multi-level feature extraction network, a Prototypes Generation (PG) module, a Prototype Localization (PL) module and a Self-Reinforcing Prototypes Generation(SRPG) module. The multi-level feature extraction network extracts features of different levels and projects them to a uniform size. Then, the PG module is employed to obtain multi-prototypes by an unsupervised Gaussian mixture model with hidden variables, which can get more comprehensive low-level and high-level information. Moreover, the PL module locates the foreground prototypes and guides the activation of foreground-related feature channels, then obtains the foreground and background probability maps. Finally, the SRPG module uses query prototypes to match query features, which can effectively capture the deep consistent features of query objects to match query features appropriately. Experimental results illustrate that the proposed MMLNet can achieve 66.33% and 35.89% mIoU with ResNet-50 backbone in the 1-shot setting of PASCAL-5i and COCO-20i without any post-processing refinement, which is the best performance in all comparable methods. It also verifies the effectiveness and performance of the proposed model.