Multi-frame Video Super-resolution Based on Efficient and Parallel Network

Multi-frame video super-resolution(VSR) aims to restore a high-resolution video from both its corresponding low-resolution frame and multiple neighboring frames, in order to make full use of the inter-frame information. However, vast computation complexity hinders the inference speed of video super-resolution. In order to increase the inference speed while ensuring the accuracy of the model, we proposed an efficient and parallel multi-frame VSR network, termed EPVSR. The proposed EPVSR is based on spatio-temporal adversarial learning to achieve temporal consistency and uses TecoGAN as the baseline model. By adding an improved non-deep network, which is composed of parallel subnetworks with multi-resolution streams, these streams are fused together at regular intervals to exchange information. we reduced the number of parameters and make the model lighter. Besides, we implement structural re-parameterization network acceleration technique to optimize the inference process of EPVSR network. Finally, our EPVSR achieves the real-time processing capacity of 4K@36.45FPS. compared with TecoGAN, we achieve 9.75 × performance speedups, but the effect is not reduced. the PSNR of EGVSR are increased by 3.36%. The experimental results show that the non-deep network can effectively speed up the model inference, and the proposed EPVSR has a good super-resolution effect.