Video Deblurring via Spatiotemporal Pyramid Network and Adversarial Gradient Prior

Video deblurring is to restore sharp frames from a blurry sequence. It is a challenging low-level vision task because the blur caused by camera shake, object motions and depth variations is heterogeneous in both spatial and temporal dimensions. Traditional methods usually work on a fixed spatiotemporal scale. However, the spatiotemporal scale of blurs in the video can vastly vary in the real-world situation. To address this challenge, we propose a Spatiotemporal Pyramid Network (SPN) to dynamically learn different spatiotemporal cues for video deblurring. Specifically, inside SPN, a spatiotemporal pyramid module is employed to effectively capture both spatial information and temporal information from the blurry sequence in a pyramid mode. An image reconstruction module constructs the sharp center frame through the obtained spatiotemporal information. Additionally, inspired by the statistical image prior and adversarial learning, we extend SPN and propose a Spatiotemporal Pyramid Generative Adversarial Network (SPGAN), which conducts adversarial discrimination in the gradient space. It helps the network produce more realistic sharp video frames. Experiments conducted on benchmarks demonstrate that the proposed methods achieve state-of-the-art results in terms of PSNR, SSIM and visual quality.