Towards Capsule Routing As Reconstruction With Sparsity Constraints

The most recently-proposed concept of capsule network consists of capsules, a structural group of neurons with activation, as building blocks, and dynamic routing between them as connections. Semantic information for final tasks can be extracted by stacking such capsule layers as construction of deep models. In this paper, we formulate the dynamic routing problem from the perspective of feature compression and reconstruction, proposing a novel routing algorithm which explicitly minimizes the reconstruction error with sparsity constraints. By alternately updating poses and routing weights with closed-form solutions, the method ensures a theoretical convergence of the dynamic routing. Benefitting from the strict convergence, we further prompt the efficiency of routing with the use of an adaptive number of iterations. Experimental results on different datasets prove that our proposed routing method outperforms existing routing methods on both performance as well as efficiency with obvious margin. Furthermore, we empirically show that the reconstruction based routing method can be embedded into backbone networks as a representative compression module. The integration of convolutional networks and capsule routing delivers both reduction of model parameters and improvement of performance to original backbones. (C) 2020 Elsevier B.V. All rights reserved.