Establishing Eye Contact for Home Video Communication Using Stereo Analysis and Free Viewpoint Synthesis

Eye contact has been proven to be an important visual cue in video communication applications. We present a method to re-establish eye contact in a home video communication scenario caused by misalignment between the camera and the communication window the participant looks at. Our method covers the complete algorithm chain from acquisition to rendering and uses a pixel-based 3D analysis and rendering approach to create a virtual view of a camera placed at the position of the communication window. The outcomes of a large scale subjective study identified the crucial problems of such an approach. Based on the significant observations during the study in this paper we address the most important problems. We propose a method that produces spatially consistent depth maps using cross-check based filling. We address aliasing artifacts during point rendering and present a method to enhance the virtual view by image inpainting based on robust contour warping.