Principal Semantic Feature Analysis with Covariance Attention

In this work, we present a new module for semantic segmentation. This new module is designed as a plug in module for the backbone networks to further boosting the segmentation performance using the principal semantic feature analysis with covariance attention. Specifically, the spatial and channel covariance attention module are designed respectively, which can filter noisy regions and help the CNN to adaptively extract the dominant semantic content. By using the proposed covariance attention modules, a covariance attention architecture is built over FCN. Experimental results demonstrate the substantial benefits brought by the proposed covariance attention scheme, and show that the covariance attention mechanism is feasible and effective for improving the accuracy of semantic segmentation.