Multi-Scale Adaptive Structure Network For Human Pose Estimation From Color Images

Human pose estimation is formulated as a joint heatmap regression problem in the deep learning based methods. Existing convolutional neural networks usually adopt fixed kernel size for generating joint heatmaps without regard to the size of human shapes. In this paper, we propose a novel method to address this issue by adapting the kernel size of joint heatmaps to the human scale of the input images in the training stage. We present a normalization strategy of how to perform the adaption between the kernel size and human scale. Beyond that, we introduce a novel limb region representation to learn the human pose structural information. Both the adaptive joint heatmaps as well as the limb region representation are combined together to construct a novel neural network, which is named Multi-scale Adaptive Structure Network (MASN). The effectiveness of the proposed network is evaluated on two widely used human pose estimation benchmarks. The experiments demonstrate that our approach could obtain the state-of-the-art results and outperform the most existing methods over all the body parts.