PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos
arxiv(2024)
摘要
The primary focus of Neural Representation for Videos (NeRV) is to
effectively model its spatiotemporal consistency. However, current NeRV systems
often face a significant issue of spatial inconsistency, leading to decreased
perceptual quality. To address this issue, we introduce the Pyramidal Neural
Representation for Videos (PNeRV), which is built on a multi-scale information
connection and comprises a lightweight rescaling operator, Kronecker
Fully-connected layer (KFc), and a Benign Selective Memory (BSM) mechanism. The
KFc, inspired by the tensor decomposition of the vanilla Fully-connected layer,
facilitates low-cost rescaling and global correlation modeling. BSM merges
high-level features with granular ones adaptively. Furthermore, we provide an
analysis based on the Universal Approximation Theory of the NeRV system and
validate the effectiveness of the proposed PNeRV.We conducted comprehensive
experiments to demonstrate that PNeRV surpasses the performance of contemporary
NeRV models, achieving the best results in video regression on UVG and DAVIS
under various metrics (PSNR, SSIM, LPIPS, and FVD). Compared to vanilla NeRV,
PNeRV achieves a +4.49 dB gain in PSNR and a 231
with a +3.28 dB PSNR and 634
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要