Low Pixel Rate 3DoF+ Video Compression Via Unpredictable Region Cropping

Enhanced three degrees of freedom (3DoF+) video system enables both rotational and translational movements for viewers within a limited scene. It introduces interactive motion parallax that provides viewers with a more natural and immersive visual experience on head-mounted displays (HMDs). A large set of views are required to support 3DoF+ video system, hence huge data costs a tremendous amount of computation and bandwidth. In this paper, a method based on unpredictable region cropping is proposed to reduce the size of data. The proposed method can be separated into two steps: basic view selection and sub-image cropping. One or several views in the source view set are selected as basic views adaptively. The goal of the basic views is to predict the remaining views, and unpredictable regions are cropped to multiple rectangular sub-images. The basic views and the cropped sub-images, called to-be-coded views, are coded by using High Efficiency Video Coding (HEVC) standard. Experimental results show the proposed method can achieve up to 75.0% pixel-rate saving and improve the quality of rendered views by 1.5dB. The method of basic view selection was adopted by the moving picture experts group (MPEG) video subgroup as a tool for the first version of Test Model of Immersive Video (TMIV).