Images captured under low-light conditions pose significant limitations in many applications. Removing the illumination effects and enhancing the quality of such images is crucial. In this paper, we introduce a low-light image enhancement variational method based on the Retinex decomposition into illumination, reflectance, and noise. Our model integrates a novel nonlocal gradient-type fidelity term designed to preserve structural details. Additionally, we propose an automatic gamma correction module. Since the noise is explicitly addressed, no post-processing is required for denoising. Experimental results show that our method compares favorably with state-of-the-art deep learning-based approaches.

Images captured under low-light conditions present significant limitations in many applications, as poor lighting can obscure details, reduce contrast, and hide noise. Removing the illumination effects and enhancing the quality of such images is crucial for many tasks, such as image segmentation and object detection. In this paper, we propose a variational method for low-light image enhancement based on the Retinex decomposition into illumination, reflectance, and noise components. A color correction pre-processing step is applied to the low-light image, which is then used as the observed input in the decomposition. Moreover, our model integrates a novel nonlocal gradient-type fidelity term designed to preserve structural details. Additionally, we propose an automatic gamma correction module. Building on the proposed variational approach, we extend the model by introducing its deep unfolding counterpart, in which the proximal operators are replaced with learnable networks. We propose cross-attention mechanisms to capture long-range dependencies in both the nonlocal prior of the reflectance and the nonlocal gradient-based constraint. Experimental results demonstrate that both methods compare favorably with several recent and state-of-the-art techniques across different datasets. In particular, despite not relying on learning strategies, the variational model outperforms most deep learning approaches both visually and in terms of quality metrics.

In this work, we propose a variational model to solve the ill-posed inverse problem of estimating luminance and reflectance from a given observation. A theoretical analysis of the resulting energy functional is provided.

When training a neural network for object detection a great deal of effort is usually devoted to augment the training dataset. The rationale behind this process is that augmentation increases the generalization capability of the network. However, little attention has been paid to the application of image enhancement techniques as a pre-processing step of the training task. In this paper we show, in the context of fish detection in submarine images, that the application of classical color enhancement methods may improve significantly the performance of the well known Mask R-CNN object detector.

With the advent of deep-learning (DL) techniques, image annotation has become a fundamental part of the research process. In the case of underwater image annotation, the human in charge of the task is faced whith the inherent quality problems of this kind of images. A large number of underwater image enhancement (UIE) methods have been developed aimed at improving the colors and contrast of these images. However, no attention has been paid to the specific problem of image annotation. In this case the global image quality of the processed image is less important than the fact that the objects to be annotated stand out and that their contours are easy to delineate. In this paper we evaluate seven state-of-the-art UIE techniques and rank them, through a subjective approach, according to their utility for the annotation process. The conclusion of our study is that, in general, the model-free Multiscale Retinex algorithm is preferred over more complex techniques that try to model the formation of the underwater images.

The Retinex perception theory tries to mimic the human ability to cope with the high dynamic range of natural scenes. In 1986 E. Land proposed a formulation of this model in terms of a Center/Surround operation involving two steps, a local adaptation and a global transform. This model gave rise to the so-called Center/Surround tone-mapping algorithms. In this paper we unify the different Center/Surround algorithms proposed in the literature using a common framework and analyze several possibilities for the local and global operations involved.

The authors present a method for the enhancement of backlit images, i.e. images in which the main source of light is behind the photography subject. These images contain, simultaneously, very dark and very bright regions. In this situation, a single tone mapping function is unable to enhance the whole image. They propose the use of several such tone mappings, some of them enhancing the dark regions while others enhancing the bright regions, and then the combination of all these results using an image fusion algorithm. Qualitative and quantitative results confirm the validity of the proposed method.

The receptive fields in the Human Visual System (HVS) are organized in such a way that they not merely capture information on the photoreceptor’s exposition to light but also on the differences in firing rates of their center and surround cells. This organization can be related with the human’s ability to cope with the high dynamic range of natural scenes. As a way to mimic this behavior E. Land proposed in 1986 a perception model consisting in two steps, namely, a local adaptation followed by a global transform. This model gave rise to the so-called Center/Surround tone-mapping algorithms, which are used to map the intensity values of high dynamic range (HDR) images to the limited range of displayable images (typically using 8-bits per channel). In this paper we unify the different Center/Surround algorithms proposed in the literature using a common framework and analyze several possibilities for the local and global operations involved. We accompany our study with quantitative and qualitative results that permit us to suggest the best pair of local/global transforms for a Center/Surround method.

We propose a novel algorithm for multi-exposure fusion (MEF). This algorithm decomposes image patches with the DCT transform. Coefficients from patches with different exposure are combined. The luminance and chrominance of the different images are fused separately. The algorithm adapts to dynamic sequences in order to avoid ghosting effects. The initial sequence is processed to be made static before applying the fusion procedure. Experiments with several data sets show that the proposed algorithm performs better than state-of-the-art.

Most satellites decouple the acquisition of a panchromatic image at high spatial resolution from the acquisition of a multispectral image at lower spatial resolution. Pansharpening is a fusion technique used to increase the spatial resolution of the multispectral data while simultaneously preserving its spectral information. In this paper, we consider pansharpening as an optimization problem minimizing a cost function with a nonlocal regularization term. The energy functional which is to be minimized decouples for each band, thus permitting the application to misregistered spectral components. This requirement is achieved by dropping the, commonly used, assumption that relates the spectral and panchromatic modalities by a linear transformation. Instead, a new constraint that preserves the radiometric ratio between the panchromatic and each spectral component is introduced. An exhaustive performance comparison of the proposed fusion method with several classical and state-of-the-art pansharpening techniques illustrates its superiority in preserving spatial details, reducing color distortions, and avoiding the creation of aliasing artifacts.

We present a generalized white-patch technique able to rapidly detect color cast of natural images. Instead of relying on the chromatic information of a single perfectly reflective patch in the image, as pure white-patch models do, we consider a connected region of pixels that will serve as white reference for the method. The pixels belonging to the white reference region must comply with three properties: 1) they do not have to be completely saturated; 2) they must belong to the p% of pixels with brightest intensity in the whole image (where p is a parameter of the model); 3) the area of the connected region formed by these pix-els must overcome a threshold of significance A (a second parameter). Color cast is detected if the average intensity in the three separated chromatic channels RGB is distant enough from a neutral grey level, where the distance is measured through an angular metric.

Common satellite imagery products consist of a panchromatic image at high spatial resolution and several misregistered spectral bands at lower resolution. Pansharpening is the fusion process by which a high-resolution multispectral image is inferred. We propose a variational model for which pan-sharpening is defined as an optimization problem minimizing a cost function with nonlocal regularization. We incorporate a new term preserving the radiometric ratio between the panchromatic and each spectral band. The resulting model is channel-decoupled, thus permitting the application to misregistered spectral data. The experimental results illustrate the superiority of the proposed method to preserve spatial details, reduce color artifacts, and avoid aliasing.

Enhancement algorithms are absolutely necessary for the visualization of both shadowed and bright image regions. Defining algorithms that permit to visualize them simultaneously without altering the image content is therefore extremely relevant for remote sensing applications. In this paper, we present the results of two successive benchmarks which tested the performance of the state-of-the-art contrast enhancement and tone-mapping algorithms applied to satellite images. Experts from the French Space Agency Centre National d'Etudes Spatiales (CNES), Service Regional de Traitement d'Image et de Teledetection (SERTIT), and two European universities assessed the quality and fidelity of the results of several state-of-the-art enhancement algorithms on the excerpts from seven images (five Pleiades and two simulated 30-cm images). The first benchmark permitted to tighten the procedure and the selection of the test images for the second one, and to make a first selection of concurrent algorithms. The second benchmark not only included the best algorithms selected by the first benchmark but also added even more competitors in the tone-mapping class. The results of both benchmarks were coherent. They point a particular retinex-based algorithm as the best compromise between the competitive requirements of a contrast enhancement in dark regions and a preservation of detail in bright parts.

This paper deals with the analysis, implementation, and comparison of several vector-valued total variation (TV) methods that extend the Rudin-Osher-Fatemi variational model to color images. By considering the discrete gradient of a multichannel image as a 3D structure matrix with dimensions corresponding to the spatial extend, the differences to other pixels and the color channels, we introduce in [J. Duran, M. Moeller, C. Sbert, and D. Cremers, 'Collaborative Total Variation: A General Framework for Vectorial TV Models', SIAM Journal on Imaging Sciences, 9(1), pp.116-151, 2016] collaborative sparsity enforcing norms for penalizing the resulting tensor. We call this class of regularizations collaborative total variation (CTV). We first analyze the denoising properties of each collaborative norm for suppressing color artifacts while preserving image features and aligning edges. We then describe the primal-dual hybrid gradient method for solving the minimization problem in detail. The resulting CTV–L2 variational model can successfully be applied to many image processing tasks. On the one hand, an extensive performance comparison of several collaborative norms for color image denoising is provided. On the other hand, we analyze the ability of different CTV methods for decomposing a multichannel image into a cartoon and a textural part. Finally, we also include a short discussion on alternative minimization methods and compare their computational efficiency.

Even after two decades, the total variation (TV) remains one of the most popular regularizations for image processing problems and has sparked a tremendous amount of research, particularly on moving from scalar to vector-valued functions. In this paper, we consider the gradient of a color image as a three-dimensional matrix or tensor with dimensions corresponding to the spatial extent, the intensity differences between neighboring pixels, and the spectral channels. The smoothness of this tensor is then measured by taking different norms along the different dimensions. Depending on the types of these norms, one obtains very different properties of the regularization, leading to novel models for color images. We call this class of regularizations collaborative total variation (CTV). On the theoretical side, we characterize the dual norm, the subdifferential, and the proximal mapping of the proposed regularizers. We further prove, with the help of the generalized concept of singular vectors, that an $\ell^{\infty}$ channel coupling makes the most prior assumptions and has the greatest potential to reduce color artifacts. Our practical contributions consist of an extensive experimental section, where we compare the performance of a large number of collaborative TV methods for inverse problems such as denoising, deblurring, and inpainting.

Image restoration is the problem of recovering an original image from an observation of it in order to extract the most meaningful information. In this paper, we study this problem from a variational point of view through the minimization of energies composed of a quadratic data-fidelity term and a nonsmooth nonconvex regularization term. In the discrete setting, existence of minimizer is proved for arbitrary linear operators. For this kind of problems, fully segmented solutions can be found by minimizing objective nonconvex functionals. We propose a dual formulation of the model by introducing an auxiliary variable with a double function. On one hand, it marks the edges and it ensures their preservation from smoothing. On the other hand, it makes the criterion half-linear in the sense that the dual energy depends linearly on the gradient of the image to be recovered. This leads to design an efficient optimization algorithm with wide applicability to several image restoration tasks such as denoising and deconvolution. Finally, we present experimental results and we compare them with TV-based image restoration algorithms.

In this paper, we propose a novel framework for restoring color images using nonlocal total variation (NLTV) regularization. We observe that the discrete local and nonlocal gradient of a color image can be viewed as a 3D matrix/or tensor with dimensions corresponding to the spatial extend, the differences to other pixels, and the color channels. Based on this observation we obtain a new class of NLTV methods by penalizing the ℓ p,q,r norm of this 3D tensor. Interestingly, this unifies several local color total variation (TV) methods in a single framework. We show in several numerical experiments on image denoising and deblurring that a stronger coupling of different color channels – particularly, a coupling with the ℓ ∞ norm – yields superior reconstruction results.