Sparse Shape Encoding for Topologically Improved Instance Segmentation

Neurophysiological studies suggest that neurons in the intermediate visual area V4 of the primate cortex encode a sparse representation of object shape. While there are metabolic arguments for such sparse representations, there are also potential advantages for inference. Here we explore whether sparse shape encoding can yield benefits for object instance segmentation. Specifically, we encode 2D object shape using a Distance Transform Map (DTM) and learn a sparse basis for this representation. To make use of this encoding, we design and train an instance segmentation head to estimate the sparse coefficients representing the shape of each object, and then recover the estimated shape from the zero-crossing level set of the corresponding DTM. Our novel SparseShape encoding approach produces fewer topological errors than the state of the art, yields competitive mask AP on the COCO benchmark and exhibits superior generalization performance on the Cityscapes instance segmentation task. These results suggest that the sparse shape encoding observed in primate cortex has computational advantages that can benefit computer vision instance segmentation systems.