IOP-CapsNet with ISEMRA: Fetching part-to-whole topology for improving detection performance of articulated instances

Due to the unique focus on positional understanding, Capsule Network (CapsNet) has received sufficient attention in the field of computer vision. However, since low-level capsules can vote in favor of every high-level capsule irrespective of their interrelationship, such blindly fully-connected communication mode suffers from misassignment. To tackle this issue, we propose to constrain the bottom-up voting scope by quantifying such correlations, hoping the unrelated or weakly related decision paths are cut off. This paper formulas such pipeline as an Inside-Out Perception Capsule Network (IOP-CapsNet) with Interrelationship-Steered Expectation -Maximum Routing-by-Agreement (ISEMRA). The part-whole relationships fetched by IOP-CapsNet is also used to implement accurate object detection of articulated instances. Three constraints, dubbed Intra-Object Cohesive-ness Quantification (IOCQ), Part Backtracking (PB), and Vote Screening (VS), are customized and embedded into ISEMRA to restrain the voting scope from the perspectives of intra-object cohesiveness and external association. They stipulate that only these primary capsules (parts) satisfying the criteria of both internal consistency and external association are permitted to update entity capsules (the whole/composites). As a result, firstly, an entire instance can be separated into several representative parts; and secondly, the part-object relationships among split parts are refined during bottom-up "component backtracking" procedure for object detection. ISEMRA enables high-level capsules to optionally aggregate projection from non-spatially-fixed sets of low-level capsules. Quantitative and ablation verifications on VOC2007, VOC2012, OICOD18, ILSVRC17, and COCO18 datasets show the superiority of IOP-CapsNet over the state-of-the-art models.