Interactive 360^∘ Video Streaming Using FoV-Adaptive Coding with Temporal Prediction
CoRR(2024)
摘要
For 360^∘ video streaming, FoV-adaptive coding that allocates more
bits for the predicted user's field of view (FoV) is an effective way to
maximize the rendered video quality under the limited bandwidth. We develop a
low-latency FoV-adaptive coding and streaming system for interactive
applications that is robust to bandwidth variations and FoV prediction errors.
To minimize the end-to-end delay and yet maximize the coding efficiency, we
propose a frame-level FoV-adaptive inter-coding structure. In each frame,
regions that are in or near the predicted FoV are coded using temporal and
spatial prediction, while a small rotating region is coded with spatial
prediction only. This rotating intra region periodically refreshes the entire
frame, thereby providing robustness to both FoV prediction errors and frame
losses due to transmission errors. The system adapts the sizes and rates of
different regions for each video segment to maximize the rendered video quality
under the predicted bandwidth constraint. Integrating such frame-level FoV
adaptation with temporal prediction is challenging due to the temporal
variations of the FoV. We propose novel ways for modeling the influence of FoV
dynamics on the quality-rate performance of temporal predictive coding.We
further develop LSTM-based machine learning models to predict the user's FoV
and network bandwidth.The proposed system is compared with three benchmark
systems, using real-world network bandwidth traces and FoV traces, and is shown
to significantly improve the rendered video quality, while achieving very low
end-to-end delay and low frame-freeze probability.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要