Application of 3D Human Pose Estimation for Behavioral Reproduction.

Human pose estimation is a challenging problem in computer vision and shares all the difficulties of object detection. This task is particularly problematic when it comes to the estimation of the human pose in three dimensions. In fact, while the perception of an object in three dimensions is easy for humans due to the mastering of 3D mental model through years, this task is harder to replicate in computer vision. In this paper, we describe an approach that aims at estimating poses from video with the objective of reproducing the observed behaviors by a virtual avatar. We are motivated by two main objectives. First, we aim at the estimation of 3D poses in video. We use a fully convolutional model based on temporal (dilated) convolutions over 2D keypoints to achieve the estimation of these 3D poses. Secondly, we set the objective to transfer 3D coordinates for previously estimated poses to a virtual avatar. The idea behind this transfer is to reproduce observed behavior in a video by a virtual avatar. Our strategy of employing temporal convolutions in lifting 2D keypoints to 3D keypoints yields better results than previous methods that attempt similar reconstruction. Finally, we use an image-based position transfer to recreate the behaviors derived from the video on the skeleton of a virtual avatar. With our approach we create learning avatar that could be used in different applications.