360° video quality assessment based on saliency-guided viewport extraction

Due to the distortion of projection generated during the production of 360^∘ video, most quality assessment algorithms used for 2D video have the problem of performance degradation. In this paper, we propose a full-reference 360^∘ video quality assessment method, utilizing saliency to guide viewport extraction to eliminate the projection distortion. To be more specific, we first predict the visual saliency of each frame with a 360^∘ saliency prediction network and then select the viewport that optimally represents the video frame through the optimal viewport positioning module (OVPM). Furthermore, we propose the attention-based three-dimensional convolutional neural network (3D CNN) quality assessment network to evaluate the video quality, in which 3D CNN convolution and attention modules can better capture the quality degradation of distorted viewports. Experimental results show that our method achieves superior performance in 360^∘ video quality assessment tasks.