Real-Time Human Body Correspondences For Motion Tracking

Non-rigid human surface tracking systems have many important applications in virtual reality and mixed reality. However, current systems can hardly be applied to interactive scenarios for their performance or accuracy limitations. One of the biggest challenges is efficiently estimating accurate correspondences between the template human model and tracking data. To bridge the gap, we propose a hierarchical feature matching framework for computing accurate dense correspondences between human shapes in real-time. For input human mesh model and depth scan, we train a fully convolutional network to produce dense feature descriptors with local similarity on human surface. Base on that, correspondences are found by performing fast hierarchical matching on segmented human body. Our approach is robust to large motion and deformation, its efficiency is validated in related real-time scenarios, while its accuracy has been proven comparable to state-of-the-art offline methods.