Video deinterlacing remains a significant challenge due to structural artifacts and information loss. When displayed on modern digital devices, early interlaced videos often suffer from complex interlacing and compression artifacts, which severely degrade visual quality. Existing deinterlacing methods typically struggle to handle such diverse artifacts while preserving fine-grained details. To address these issues, we propose a novel Spatiotemporal Synchronization for Video Deinterlacing (SSVD). First, we design a Multi-Directional Shuffling Module (MDSM) to enhance the model's ability to capture spatial dependencies, thereby guiding the prediction of missing fields. Second, a Dynamic Cross-frame Interaction Module (DCIM) is incorporated to implicitly model inter-frame correspondences, effectively leveraging cross-frame information to alleviate the blurring and artifacts. Third, we develop a Gated Refinement Module (GRM) to achieve fine-grained reconstruction. Experimental results demonstrate SSVD is superior to the state-of-the-art algorithms in video deinterlacing tasks, achieving superior visual quality and detail reconstruction. Source code will be made public after the review is completed.

Most of the existing low-light enhancement research focuses on global illumination enhancement while ignoring the issues of brightness unevenness and color distortion. To address this dilemma, we propose a low-light image enhancement method that can achieve good performance in luminance adjustment and color correction simultaneously. Specifically, the Luminance Adjustment Module is designed to model the global luminance adjustment parameters while taking into account the relationship between global and local illumination features, in order to prevent overexposure or underexposure. Furthermore, we design a Color Correction Module based on the attention mechanism, which utilizes the attention mechanism to capture global color features and correct the color deviation in the illumination-enhanced image. Additionally, we design a color loss function based on a 14-dimensional statistical feature vector related to color, enabling further restoration of the image’s true color. We conduct empirical studies on multiple public low-light datasets, demonstrating that the proposed method outperforms other representative state-of-the-art models regarding illumination enhancement and color correction.

Single image super-resolution aims to increase the size and the visual effectiveness of a low-resolution image. Although existing deep learning-based methods have achieved promising results, they still face great challenges in dealing with the reconstruction of complicated scenes. This is mainly because the deep convolution operations cannot balance the low frequency contents and high frequency details. To mitigate this problem, we propose VTSR, an image super-resolution method based on the V-transform. In VTSR, we propose a VT-block. Change the previous practice of using a convolution layer to directly increase the network dimension to , while the spatial and frequency domain information is parallel, providing richer information for subsequent deep convolution neural networks. Then, we introduce the V-transform, a wavelet-transform, to the super-resolution task to extract richer frequency information. Finally, we designed a loss function mainly for frequency domain information, which has a positive impact on the super-resolution effect and training. Experiments show that the combination of our proposed method and V-transform can achieve better results than most state-of-the-art methods.

Single Image Reflection Removal (SIRR) is a hot topic in downstream tasks of computer vision, with the aim of eliminating undesirable reflections in degraded images taken through glass. However, due to the ill-posed property of SIRR and the lack of large-scale real world mixture images contaminated by reflections, existing methods degrade on real datasets and suffer from the problem of reflection residue. In this work, we propose an efficient SIRR pyramid model that can robustly recover real images degraded by reflections. Specifically, our network uses a Nonlinear Activation Free Network (NAFNet) as a baseline, a variant of U-Net, which is capable of extracting feature maps of different scales. Furthermore, to learn the pixel-level long-distance feature correspondence between multi-scale features, a pyramid fusion module based on scale-agnostic attention has been embedded into the baseline. Comprehensive experimental results demonstrate the effectiveness of our model.

This paper reviews the video colorization challenge on the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2023. The target of this challenge is converting grayscale videos into color videos with better colorization performance and temporal consistency. The challenge consists of two tracks. For Track 1, the goal is achieving the best FID (Fréchet Inception Distance) while being constrained to maintain or improve over the baseline method in terms of the temporal-consistency metric. The Color Distribution Consistency (CDC) index is used as the temporal consistency evaluation metric in this challenge. For Track 2, the target is to obtain a solution with the best CDC result while being constrained to maintain or improve over the baseline method in terms of FID. We use DeOldify-video as the baseline method for two tracks. For the final testing phase of both tracks, six teams submitted fact sheets and executable code of their solutions. This report brings together descriptions and discussions of all these solutions. Both tracks use the same data and the datasets are available at this url.

Video colorization is a highly challenging and ill-posed problem that suffers from severe flickering artifacts and color distribution inconsistency. To resolve these issues, we propose a Restored Transformer and Test-time Local Converter network(RTTLC). Firstly, we introduce a Bidirectional Recurrent Block and a Learnable Guided Mask to our network. This leverages hidden knowledge from adjacent frames that include rich information about occlusion, resulting in significant enhancements in visual quality. Secondly, we integrate a Restored Transformer that enables the network to utilize more spatial contextual information and capture multi-scale information more accurately. Thirdly, during inference, we utilize the Test-time Local Converter(TLC) strategy to alleviate distribution shift and enhance the performance of the model. Experimental results show good performance of FID and CDC. Notably, RTTLC achieves second prize in both tracks of the NTIRE23 video colorization challenges.

High dynamic range(HDR) imaging is the task of re-covering HDR image from one or multiple input Low Dynamic Range (LDR) images. In this paper, we present Gamma-enhanced Spatial Attention Network(GSANet), a novel framework for reconstructing HDR images. This problem comprises two intractable challenges of how to tackle overexposed and underexposed regions and how to overcome the paradox of performance and complexity trade-off. To address the former, after applying gamma correction on the LDR images, we adopt a spatial attention module to adaptively select the most appropriate regions of various exposure low dynamic range images for fusion. For the latter one, we propose an efficient channel attention module, which only involves a handful of parameters while bringing clear performance gain. Experimental results show that the proposed method achieves better visual quality on the HDR dataset. The code will be available at: https://github.com/fancyicookie/GSANet

本文介绍了基于人工智能的视频超分辨率算法原理,总结了国内外视频超分辨领域的相关数据集和常用视频质量评价指标,列举了国内视频修复和增强的部分实践案例,简要分析了基于人工智能的视频修复和增强算法领域的问题.

Demoiréing is a multi-level image restoration task involving texture and color restoration. However, the moiré pattern in documents is difficult to remove, and the blurred font also appears. In this paper, we propose a multi-scale coarse-to-fine neural network to solve this problem. First, a multi-scale dense network structure is adopted to capture the moiré pattern from different resolution features, in which a spatial attention module is utilized that has remarkable performance in capturing moiré pattern. Second, a lightweight deblurring network is taken to remove the blur in documents for further optimziation. Through ablation experiments, the effectiveness of our method is demonstrated. The network proposed in this paper takes the third place in the document demoiréing track of the 2022 Baidu.com AI Competition.

Recently, with the rapid development of mobile sensing technology, capturing scene information by mobile sensing devices in the form of images or videos has become a prevalent recording method. However, the moiré pattern phenomenon may occur when the scene contains digital screens or regular strips, which greatly degrade the visual performance and image quality. In this paper, considering the complexity and diversity of moiré patterns, we propose a novel end-to-end image demoiré method, which can learn moiré pattern elimination in both the frequency and spatial domains. To be specific, in the frequency domain, considering the signal energy of moiré pattern is widely distributed in the frequency, we introduce a wavelet transform to decompose the multi-scale image features, which can help the model identify the moiré features more precisely to suppress them effectively. On the other hand, we also design a spatial domain demoiré block (SDDB). The SDDB module can extract moiré features from the mixed features, then subtract them to obtain clean image features. The combination of the frequency domain and the spatial domain enhances the model's ability in terms of moiré feature recognition and elimination. Finally, extensive experiments demonstrate the superior performance of our proposed method to other state-of-the-art methods. The Grad-CAM results in our ablation study fully indicate the effectiveness of the two proposed blocks in our method.

This paper reviews the challenge on constrained high dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2022. This manuscript focuses on the competition set-up, datasets, the proposed methods and their results. The challenge aims at estimating an HDR image from multiple respective low dynamic range (LDR) observations, which might suffer from under- or over-exposed regions and different sources of noise. The challenge is composed of two tracks with an emphasis on fidelity and complexity constraints: In Track 1, participants are asked to optimize objective fidelity scores while imposing a low-complexity constraint (i.e. solutions can not exceed a given number of operations). In Track 2, participants are asked to minimize the complexity of their solutions while imposing a constraint on fidelity scores (i.e. solutions are required to obtain a higher fidelity score than the prescribed baseline). Both tracks use the same data and metrics: Fidelity is measured by means of PSNR with respect to a ground-truth HDR image (computed both directly and with a canonical tonemapping operation), while complexity metrics include the number of Multiply-Accumulate (MAC) operations and runtime (in seconds).

Burst super-resolution has received increased attention in recent years due to its applications in mobile photography. By merging information from multiple shifted images of a scene, burst super-resolution aims to recover details which otherwise cannot be obtained using a simple input image. This paper reviews the NTIRE 2022 challenge on burst super-resolution. In the challenge, the participants were tasked with generating a clean RGB image with 4× higher resolution, given a RAW noisy burst as input. That is, the methods need to perform joint denoising, demosaicking, and super-resolution. The challenge consisted of 2 tracks. Track 1 employed synthetic data, where pixel-accurate high-resolution ground truths are available. Track 2 on the other hand used real-world bursts captured from a handheld camera, along with approximately aligned reference images captured using a DSLR. 14 teams participated in the final testing phase. The top performing methods establish a new state-of-the-art on the burst super-resolution task.

Deep learning-based burst super-resolution (SR) approaches are extensively studied in recent years, prevailing in the synthetic datasets and the real datasets. However, the existing networks rarely pay attention to the enhanced denoising problem in raw domain and they are not sufficient to restore complex texture relationships between frames. In this paper, we propose a new framework named A RAW Burst Super-Resolution Method with Enhanced Denoising (EDRBSR), which solves the BurstSR problem by jointly denoising structure and reconstruction enhancement structure. We adopt a Denoising Network to further improve the performance of noise-free SR images. Also, we propose a Reconstruction Network to enhance spatial feature representation and eliminate the influence of spatial noise. In addition, we introduce a new pipeline to compensate for lost information. Experimental results demonstrate that our method over the existing state-of-the-art in both synthetic datasets and real datasets. Furthermore, our approach takes the 5th place in synthetic track of the NTIRE 2022 Burst Super-Resolution Challenge.

We investigate some excellent algorithms in the field of video space super-resolution based on artificial intelligence, structurally analyze the network structure of the algorithm and the commonly used loss functions. We also analyze the characteristics of algorithms in the new field of video space-time super-resolution. This work helps researchers to deeply understand the video super-resolution technology based on artificial intelligence.

超高清电视技术研究和应用国家广播电视总局重点实验室联合体奥动力、江苏有线、上海海思、京东方和当虹科技等多家企业和机构,成功开展了国内首次基于有线电视网的8K超高清转播试验,实现了中国足球协会超级联赛2020赛季决赛8K实况转播.本文详细介绍了足球赛事场景下的8K超高清端到端转播方案,并详细分析了相关测试情况和试验结果.

Super-Resolution (SR) is a fundamental computer vision task that aims to obtain a high-resolution clean image from the given low-resolution counterpart. This paper reviews the NTIRE 2021 Challenge on Video Super-Resolution. We present evaluation results from two competition tracks as well as the proposed solutions. Track 1 aims to develop conventional video SR methods focusing on the restoration quality. Track 2 assumes a more challenging environment with lower frame rates, casting spatio-temporal SR problem. In each competition, 247 and 223 participants have registered, respectively. During the final testing phase, 14 teams competed in each track to achieve state-of-the-art performance on video SR tasks.

This paper reviews the NTIRE 2021 challenge on learning the super-Resolution space. It focuses on the participating methods and final results. The challenge addresses the problem of learning a model capable of predicting the space of plausible super-resolution (SR) images, from a single low-resolution image. The model must thus be capable of sampling diverse outputs, rather than just generating a single SR image. The goal of the challenge is to spur research into developing learning formulations and models better suited for the highly ill-posed SR problem. And thereby advance the state-of-the-art in the broader SR field. In order to evaluate the quality of the predicted SR space, we propose a new evaluation metric and perform a comprehensive analysis of the participating methods. The challenge contains two tracks: 4× and 8 scale factor. In total, 11 teams competed in the final testing× phase.

Relighting is an interesting yet challenging low-level vision problem, which aims to re-render the scene with new light sources. In this paper, we introduce LTNet, a novel framework for image relighting. Unlike previous methods, we propose to solve this challenging problem by decoupling the enhancement process. Specifically, we propose to train a network that focuses on learning light variations. Our key insight is that light variations are the critical information to be learned because the scene stays unchanged during the light transfer process. To this end, we employ a global residual connection and corresponding residual loss for capturing light variations. Experimental results show that the proposed method achieves better visual quality on the VIDIT dataset in the NTIRE2021 relighting challenge.