This paper deals with the application of a geometric setting widely employed in theoretical physics, namely the fiber bundles, to color image restoration. The key idea of this approach is to model an image as a function on a principal bundle satisfying an equivariance property with respect to the action of a Lie group acting on its pixel values, and which can model the lighting changes in a scene. In this context, a natural tool for the differentiation of an image is by means of a covariant derivative. In previous works, optimal covariant derivatives have been constructed as solutions of a variational model consisting of the minimization of the L2 norm of the covariant derivative of the image, and applied to various tasks in color image restoration through the extension of the Total Variation regularizer to vector bundles. The aim of this paper is to extend these works by constructing optimal second order covariant derivatives as solutions of the minimization of the norm of the second order covariant derivative of the image. Experiments on deblurring and super-resolution corroborate the relevance of the proposed model for color image restoration. More generally, this paper validates the use of the geometric setting of fiber bundles in imaging sciences.

From their inception in the 1950s, artificial neural networks (ANNs) started using the so-called point neuron model then prevalent in neuroscience, hoping that this analogy would allow for a better emulation of brain function. Over the years the neuroscience literature has shown that the point neuron model is too simplistic to properly represent many fundamental neural processes; however, the standard neuron model in ANNs still remains the same. Here we substitute it by a very recent model of cortical cells and demonstrate through theoretical analyses and experimental results how, simply by using a more realistic neural unit element without augmenting the number of parameters, the resulting ANNs offer a number of important advantages that include increases in expressivity, robustness and learning speed, and a reduction in memorization and the amount of training data needed.

In this paper, we introduce a new variational model for color image restoration, called DIP-VBTV, which combines two priors: a deep image prior (DIP), which assumes that the restored image can be generated through a neural network, and a vector bundle total variation (VBTV), which generalizes the vectorial total variation (VTV) on vector bundles. VBTV is determined by a geometric triplet: a Riemannian metric on the base manifold, a covariant derivative, and a metric on the vector bundle. Whereas the VTV prior encourages the restored images to be piecewise constant, the VBTV prior encourages them to be piecewise parallel with respect to a covariant derivative. For well-chosen geometric triplets, we show that the minimization of VBTV encourages the solutions of the restoration model to share some visual content with the clean image. Then, we show in experiments that DIP-VBTV benefits from this property by outperforming DIP-VTV and state-ofthe-art unsupervised methods. It demonstrates the relevance of combining DIP and VBTV priors.

Variational models for inverse problems are mainly based on the choice of the regularizer, whose goal is to give the solutions some desirable property. Vectorial Total Variation, one of the most popular regularizer for color image restoration, is induced by the Euclidean gradient operator. In this paper, we introduce a new regularizer for color image restoration, induced by a nonlinear extension of the Dirac operator to color images. Whereas the Vectorial Total Variation only promotes piece-wise constant solutions, the regularizer induced by the proposed Dirac operator also promotes solutions having the gradients of their color components aligned, which turns out to be a property of natural images. Then, we insert this regularizer into a variational model for image restoration, and we approximate its numerical solution by adapting the primal-dual algorithm of convex optimization. Experiments on denoising and deblurring show that the proposed Dirac operator provides a better regularizer than the Euclidean operator.

Variational models for inverse problems are mainly based on the choice of the regularizer, whose goal is to give the solutions some desirable property. Vectorial Total Variation, one of the most popular regularizer for color image restoration, is induced by the Euclidean gradient operator. In this paper, we introduce a new regularizer for color image restoration, induced by a nonlinear extension of the Dirac operator to color images. Whereas the Vectorial Total Variation only promotes piece-wise constant solutions, the regularizer induced by the proposed Dirac operator also promotes solutions having the gradients of their color components aligned, which turns out to be a property of natural images. Then, we insert this regularizer into a variational model for image restoration, and we approximate its numerical solution by adapting the primal-dual algorithm of convex optimization. Experiments on denoising and deblurring show that the proposed Dirac operator provides a better regularizer than the Euclidean operator.

Implicit representations such as Neural Radiance Fields (NeRFs) have become a de facto standard in the field of novel view synthesis for 3D scenes. However, their stunning results typically imply the use of dozens of training images, with their corresponding cameras well localized in the scene. This paper studies new, total variation-based regularization approaches to train NeRFs in the context of very few (less than 10) training images. It leverages the NeRF back-propagation algorithm to evaluate first-order and second-order derivatives terms on the inferred depth map to enforce smoothness on the scene underlying surfaces. Through state-of-the-art performance on standard real-images benchmarks, we show that the proposed methods, coined as TV-NeRF and TGV-NeRF, make strong baselines in novel view synthesis with few training views.

Variational models for inverse problems are mainly based on the choice of the regularizer, whose goal is to give the solutions some desirable property. Total Variation, one of the most popular regularizer for image restoration, is induced by the Euclidean gradient operator and promotes piece-wise constant solutions. In this paper, we present a new regularizer for color image restoration, which is induced by a generalization of the Dirac operator. This new regularizer also encourages the gradients of the three color components of the solutions to be aligned, which is actually a property of natural images. This property is also encoded when the regularizer is induced by a Riemannian gradient, for a well-chosen Riemannian metric, but with a different mathematical formulation. Then, we compare the different regularizers by combining them with the Deep Image Prior model, this latter assuming that the restored image is the output of a neural network. Experiments on denoising and deblurring show that the proposed Dirac operator provides better results than the Euclidean and Riemannian gradient operators.

In a recent paper [T. Batard, G. Haro, and C. Ballester, SIAM J. Imag. Sci., 14 (2021), pp. 1816-1847], a new prior for image restoration was introduced. It relies first on the observation that an image and a degraded version of it can share some visual content and then on the conjecture that an image restoration model can benefit from the use of an image prior encoding this invariance property. This prior considers the restored image as a parallel section of a connection (also called covariant derivative), this latter being a critical point of an energy associated to the Lie group R+* x SO(2) acting on image pixel values. In this paper, we propose a twofold generalization of this result. First, we consider other Lie groups acting on image pixels, yielding new optimal connections. Then, we derive a family of alpha-connections from the optimal connections. The corresponding parallel sections describe new invariance properties which we use as priors encoded as penalty terms in variational models for image restoration. Experiments conducted on color image deblurring show that the proposed generalization of the work of Batard, Haro, and Ballester outperforms the original approach.

Reproducing the perception of a real-world scene on a display device is a very challenging task which requires the understanding of the camera processing pipeline, the display process, and the way the human visual system processes the light it captures. Mathematical models based on psychophysical and physiological laws on color vision, named Retinex, provide efficient tools to handle degradations produced during the camera processing pipeline like the reduction of the contrast. In particular, Batard and Bertalmío (in J Math Imaging Vis 60(6):849–881, 2018) described some psychophysical laws on brightness perception as covariant derivatives, included them into a variational model, and observed that the quality of the color image correction is correlated with the accuracy of the vision model it includes. Based on this observation, we postulate that this model can be improved by including more accurate data on vision with a special attention on visual neuroscience here. Then, inspired by the presence of neurons responding to different visual attributes in the area V1 of the visual cortex as orientation, color or movement, to name a few, and horizontal connections modeling the interactions between those neurons, we construct two variational models to process both local (edges, textures) and global (contrast) features. This is an improvement with respect to the model of Batard and Bertalmío as the latter cannot process local and global features independently and simultaneously. Finally, we conduct experiments on color images which corroborate the improvement provided by the new models.

Obtaining 3D geometry from images is a well studied problem by the computer vision community. In the concrete case of a single image, a considerable amount of prior knowledge is often required to obtain plausible reconstructions. Recently, deep neural networks in combination with 3D morphable models (3DMM) have been used in order to address the lack of scene information, leading to more accurate results. Nevertheless, the losses employed during the training process are usually a linear combination of terms where the coefficients, also called hyperparameters, must be carefully tuned for each dataset to obtain satisfactory results. In this work we propose a hyperparameters-free loss that exploits the geometry of the problem for learning 3D reconstruction from a single image. The proposed formulation is not dataset dependent, is robust against very large camera poses and jointly optimizes the shape of the object and the camera pose.

This work proposes novel hyperparameter-free losses for single view 3D reconstruction with morphable models (3DMM). We dispense with the hyperparameters used in other works by exploiting geometry, so that the shape of the object and the camera pose are jointly optimized in a sole term expression. This simplification reduces the optimization time and its complexity. Moreover, we propose a novel implicit regularization technique based on random virtual projections that does not require additional 2D or 3D annotations. Our experiments suggest that minimizing a shape reprojection error together with the proposed implicit regularization is especially suitable for applications that require precise alignment between geometry and image spaces, such as augmented reality. We evaluate our losses on a large scale dataset with 3D ground truth and publish our implementations to facilitate reproducibility and public benchmarking in this field.

In this paper, we establish a connection between image processing, visual perception, and deep learning by introducing a mathematical model inspired by visual perception from which neural network layers and image processing models for color correction can be derived. Our model is inspired by the geometry of visual perception and couples a geometric model for the organization of some neurons in the visual cortex with a geometric model of color perception. More precisely, the model is a combination of a Wilson-Cowan equation describing the activity of neurons responding to edges and textures in the area V1 of the visual cortex and a Retinex model of color vision. For some particular activation functions, this yields a color correction model which processes simultaneously edges/textures, encoded into a Riemannian metric, and the color contrast, encoded into a nonlocal covariant derivative. Then, we show that the proposed model can be assimilated to a residual layer provided that the activation function is nonlinear and to a convolutional layer for a linear activation function. Finally, we show the accuracy of the model for deep learning by testing it on the MNIST dataset for digit classification.

Human perception involves many features like contours, shapes, textures, and colors to name a few. Whereas several geometric models for contours, shapes and textures perception have been proposed, the geometry of color perception has received very little attention, possibly due to the fact that our perception of colors is still not fully understood. Nonetheless, there exists a class of mathematical models, gathered under the name Retinex, which aim at modeling the color perception of an image, which are inspired by psychophysical/physiological knowledge about color perception, and which can geometrically be viewed as the averaging of perceptual distances between image pixels. Some of the Retinex models turn out to be associated with an efficient image processing technique for the correction of camera output images. The aim of this paper is to show that this image processing technique can be improved by including more properties of the human visual system. To that purpose, we first present a generalization of the perceptual distance between image pixels by considering the parallel transport map associated with a covariant derivative on a vector bundle, from which can be derived a new image processing model for color images correction. Then, we show that the family of covariant derivatives constructed in Batard and Sochen (J Math Imaging Vis 48(3):517–543 2014 ) can model some color appearance phenomena related to brightness perception. Finally, we conduct experiments in which we show that the image processing techniques induced by these covariant derivatives outperform the original approach.

Image denoising has been a topic extensively investigated over the last three decades and, as repeatedly shown in this book, denoising algorithms have become incredibly good, so much so that many researchers have started questioning the need to further pursue this line of research. In this chapter, we argue that there is indeed room for improvement of denoising results, and we propose three different avenues to explore, none of which requires the development of new denoising methods. First, we describe how it can be better to denoise a transform of the noisy image rather than denoise the noisy image directly. We mention several possible transforms, and an open problem is to find a transform that is optimal for denoising, according to a proper image quality metric. Next, we point out the importance of having a proper noise model for JPEG pictures, so that a variance stabilization transform can be developed that transforms noise in JPEG images into additive white Gaussian noise, enabling existing denoising methods to be properly applied to the JPEG case. Finally, we highlight the fact that while virtually all denoising methods are optimized and validated in terms of the PSNR or SSIM measures, these metrics are not well correlated with perceived image quality, and therefore, it could be best to optimize the parameter values of denoising methods according to subjective testing. A remaining challenge is to develop perceptually based image quality metrics that match observer preference.

The Wilson-Cowan equations were originally proposed to describe the low-level dynamics of neural populations (Wilson&Cowan 1972). These equations have been extensively used in modelling the oscillations of cortical activity (Cowan et al. 2016). However, due to their low-level nature, very few works have attempted connections to higher level psychophysics (Herzog et al. 2003, Hermens et al. 2005) and, to the best of our knowledge, they have not been used to predict contrast response curves or subjective image quality. Interestingly (Bertalmío&Cowan 2009) showed that Wilson-Cowan models may lead to (high level) color constancy. Moreover, these models may have positive statistical effects similarly to Divisive Normalization, which is the canonical choice to understand contrast response (Watson&Solomon 1997, Carandini&Heeger 2012): while Divisive Normalization reduces redundancy due to predictive coding (Malo&Laparra 2010), Wilson-Cowan leads to local histogram equalization (Bertalmío 2014), another route to increase channel capacity. Here we show that the functional (statistical) similarities between Wilson-Cowan and Divisive Normalization actually hold and may be extended to contrast perception. Specifically, first we fitted the Wilson-Cowan model using a procedure reported for Divisive Normalization: following (Watson&Malo 2002, Laparra&Malo 2010), we maximized the correlation with human opinion in quality assessment. Secondly, we used the resulting model to predict the visibility of textured patterns on top of backgrounds of different frequencies and contrasts as in classical masking experiments. Finally, we checked the redundancy reduction of Wilson-Cowan and Divisive Normalization in the same way (as in Malo&Laparra 2010). Results show that (1) Wilson-Cowan is as good as Divisive Normalization in reproducing image distortion psychophysics, (2) Wilson-Cowan dynamics induces saturating responses that attenuate with the contrast of the background, particularly when the background resembles the test; and (3) mutual information between V1-like responses after the Wilson-Cowan interaction decreases similarly as in Divisive Normalization. Meeting abstract presented at VSS 2017

Computational design is one of the most common tasks of immersive computer graphics projects, such as games, virtual reality and special effects. Layout planning is a challenging phase of architectural design, which requires optimization across several conflicting criteria. We present an interactive layout solver that assists designers in layout planning by recommending personalized space arrangements based on architectural guidelines and user preferences. Initialized by the designer's high-level requirements, an interactive evolutionary algorithm is used to converge on an ideal layout by exploring the space of potential solutions. The major contributions of our proposed approach are addressing subjective aspects of the design to generate personalized layouts; and the development of a genetic algorithm with a multi-parental recombination method that improves the chance of generating higher quality offspring. We demonstrate the ability of our method to generate feasible floor plans which are satisfactory, based on spatial quality metrics and designer's taste. The results show that the presented framework can measurably decrease planning complexity by producing layouts which exhibit characteristics of human-made design. (C) 2017 Elsevier Inc. All rights reserved.

For helping us deliver timely decisions to our authors, the Editor-in-Chief and Publisher would like to thank the following individuals who contributed their reviews between January 1, 2016 and December 31, 2016. We applaud all your efforts and dedication to the community.

Alexandre Afgoustidis Bijan Afsari Manuel Agusti-Melchor Dmitri Alekseevsky Ali H. Al-Hamdani Andrés Almansa Andreas Alpers L. Alvarez Samuel Amstutz Eric Andres Michael Ankele Erchan Aptoula Ery Arias-Castro Egil Bae Hynek Bakstein Peter Balazs Davide Barbieri Thomas Batard Martin Bauer Amir Beck Shida Beigpour Erik Bekkers Vasileios Belagiannis Alexander Belyaev Miri Ben-Chen Rachid Benslimane Ronny Bergmann Bejamin Berkels Partha Bhowmick Isabelle Bloch Silvia Bonettini Nicolas Bonneel Ugo Boscain Petra Bosilj Matias Bossa Sven Bossuyt Jérôme Boulanger Mireille Boutin Kristian Bredies Michael Breuß Jesus Briales Martins Bruveris Martin Buhmann Martin Burger Nathan Cahill Luca Calatroni Aaron Carass Antonin Chambolle Stanley Chan Maria Charina Benjamin Charlier Caroline Chaux Beijing Chen Caihua Chen Da Chen Ke Chen Yunjin Chen Emilie Chouzenoux Giovanna Citti David Coeurjolly David Cohen-Steiner Lidija Čomić Michel Couprie Nicolas Courty Jean Cousty Xiubin Dai Juan Carlos De Los Reyes Rafael H. C. de Melo Adreano DeCesaro Tom dela Hajie Julie Delon Loic Denis Henri Der Sarkissian Agnes Desolneux Gema Diaz-Toca Stephan Didas Eric Domenjoud Marco Donatelli Bin Dong Yiqiu Dong R. Duits Alexander Effland Martin Ehler Matthias Ehrhardt Irene Epifanio Ferran Espuny Olivier Fercoq Nivan Ferreira Massimo Ferri Javier Finat Codes Jan Henrik Fitschen Andrew Fitzgibbon Luc Florack Jan Flusser Daniel Forsberg Per-Erik Forssén Benedetta Franceschiello Annalisa Franco Oren Freifeld Ulderico Fugacci Andrea Fuster Jürgen Gall Guillermo Gallego Silvano Galliani Yann Gavet Thierry Géraud Bastian Goldlücke Nuno Gonçalves Rocio Gonzalez-Diaz Joana Grah Lewis Griffin Christine Guillemot