Segmentation over Detection via Optimal Sparse Reconstructions

This paper addresses the problem of semantic segmentation, where the possible class labels are from a pre-defined set. We exploit top-down guidance, i.e. the coarse localization of the objects and their class labels, provided by object detectors. For each detected bounding box figure-ground segmentation is performed and the final result is achieved by merging the figureground segmentations. The main idea of the proposed approach, which is presented in our preliminary work [1], is to reformulate the figure-ground segmentation problem as sparse reconstruction pursuing the object mask in a non-parametric manner. The latent segmentation mask should be coherent subject to sparse error caused by intra-category diversity, thus the object mask is inferred by making use of sparse representations over the training set. In order to handle local spatial deformations, local patch-level masks are also considered and inferred by sparse representations over the spatially nearby patches. The sparse reconstruction coefficients and the latent mask are alternately optimized by applying the Lasso algorithm and the Accelerated Proximal Gradient method. The proposed formulation results in a convex optimization problem, thus the global optimal solution is achieved. In this paper we provide theoretical analysis of the convergence and optimality. We also give an extended numerical analysis of the proposed algorithm and a comprehensive comparison with the related semantic segmentation methods on the challenging PASCAL VOC object segmentation datasets and the Weizmann-Horses Dataset. The experimental results demonstrate that the proposed algorithm achieves competitive performance comparing to the state-of-the-arts.