We introduce a general framework for the reconstruction of vector-valued functions from finite and possibly noisy data, acquired through a known measurement operator. The reconstruction is done by the minimization of a loss functional formed as the sum of a convex data fidelity functional and a total-variation-based regularizer involving a suitable matrix L of differential operators. Here, the total variation is a norm on the space of vector measures. These are split into two categories: inner, and outer norms. The minimization is performed over an infinite-dimensional Banach search space. When the measurement operator is weakstar-continuous over the search space, our main result is that the solution set of the loss functional is the closed convex hull of adaptive L-splines, with fewer knots than the number of measurements. We reveal the effect of the total-variation norms on the structure of the solutions and show that inner norms yield sparser solutions. We also provide an explicit description of the class of admissible measurement operators.

Abstract. We describe a practical framework for data-driven regularization in image reconstruction. It combines model expressivity with the guarantees of variational methods. The approach is based on a weakly convex ridge regularizer, defined as the composition of a convolutional filter bank and pointwise potentials constrained to be weakly convex. These potentials are implemented as learnable splines with a strict control of their weak-convexity modulus. The resulting denoisers outperform classic convex regularization techniques as well as competitive benchmarks such as BM3D, while they still correspond to the minimization of a convex energy. The learned regularizers further extend to general inverse problems with provable convergence to critical points. Overall, this framework shows that a controlled relaxation of convexity enables the design of learnable priors that achieve strong empirical performance while preserving mathematical guarantees.

For finite-dimensional linear inverse problems where the variables are Gaussian, it is well-known that the minimum-mean-square error estimator takes the form of a regularized least-squares data fit. In this chapter, we show that this equivalence extends to a much broader infinite-dimensional setting where generalized splines take the role of linear regressors and generalized Gaussian processes on a nuclear space S are the counterpart of Gaussian random vectors. The scope of this extension is of the same nature as the switch from the classic notion of function to that of a distribution, also known as a "generalized function." Our formalism involves a whitening/regularization operator L: S→ S' whose continuous extension induces a native Hilbert space H⊂ S' that plays a central role in our characterization. The presentation is self-contained for the most part and remarkably general and powerful. It allows for the recovery of all known instances of such equivalences; in particular, the methods involving innovations and reproducing-kernel Hilbert spaces developed by Kailath and his students, and the mathematical correspondence between fractional splines and Mandelbrot's fractional Brownian motion (fractals), with the former being the optimal estimators of the latter. It also covers general Bayesian methods for the resolution of infinite-dimensional inverse problems.

We introduce the multivariate fields of experts, a new framework for the learning of image priors. Our model generalizes existing fields of experts methods by incorporating multivariate potential functions constructed via Moreau envelopes of the l(infinity )norm. We demonstrate the effectiveness of our proposal across a range of inverse problems that include image denoising, deblurring, compressed-sensing magnetic-resonance imaging, and computed tomography. The proposed approach outperforms comparable univariate models and achieves performance close to that of deep-learning-based regularizers while being significantly faster, requiring fewer parameters, and being trained on substantially fewer data. In addition, our model retains a high level of interpretability due to its structured design. It is supported by theoretical convergence guarantees which ensure reliability in sensitive reconstruction tasks.

We propose a method to efficiently compute tomographic projections of a 3D volume represented by a linear combination of shifted B-splines. To do so, we propose a ray-tracing algorithm that computes 3D line integrals with arbitrary projection geometries. One of the components of our algorithm is a neural network that computes the contribution of the basis functions efficiently. In our experiments, we consider well-posed cases where the data are sufficient for accurate reconstruction without the need for regularization. We achieve higher reconstruction quality than traditional voxel-based methods.

This paper addresses the task of learning convex regularizers to guide the reconstruction of images from limited data. By imposing that the reconstruction be amplitude-equivariant, we narrow down the class of admissible functionals to those that can be expressed as a power of a seminorm. We then show that such functionals can be approximated to arbitrary precision with the help of polyhedral norms. In particular, we identify two dual parameterizations of such systems: (i) a synthesis (or atomic) form with an L-1-penalty that involves some learnable dictionary; and (ii) an analysis form with an L-infinity-penalty that involves a trainable regularization operator. After having provided geometric insights and proved that the two forms are universal, we propose an implementation that relies on a specific architecture (tight frame with a weighted L(1)penalty) that is easy to train. We illustrate its use for denoising and the reconstruction of biomedical images. We find that the proposed framework outperforms the sparsity-based methods of compressed sensing, while it offers essentially the same convergence and robustness guarantees.

We introduce Flower, a solver for linear inverse problems. It leverages a pre-trained flow model to produce reconstructions that are consistent with the observed measurements. Flower operates through an iterative procedure over three steps: (i) a flow-consistent destination estimation, where the velocity network predicts a denoised target; (ii) a refinement step that projects the estimated destination onto a feasible set defined by the forward operator; and (iii) a time-progression step that re-projects the refined destination along the flow trajectory. We provide a theoretical analysis that demonstrates how Flower approximates Bayesian posterior sampling, thereby unifying perspectives from plug-and-play methods and generative inverse solvers. On the practical side, Flower achieves state-of-the-art reconstruction quality while using nearly identical hyperparameters across various linear inverse problems.

Information propagation characterizes how input correlations evolve across layers in deep neural networks. This framework has been well studied using mean-field theory, which assumes infinitely wide networks. However, these assumptions break down for practical, finite-size networks. In this work, we study information propagation in randomly initialized neural networks with finite width and reveal that the boundary between ordered and chaotic regimes exhibits a fractal structure. This shows the fundamental complexity of neural network dynamics, in a setting that is independent of input data and optimization. To extend this analysis beyond multilayer perceptrons, we leverage recently introduced Fourier-based structured transforms, and show that information propagation in convolutional neural networks also follow the same behavior. In practice, our investigation highlights the importance of finite network depth with respect to the tradeoff between separation and robustness. We also show that fractal patterns are observed for information propagation in the backward pass, i.e., backpropagation from the last to the first layer of finite-size networks.

We propose a statistical benchmark for diffusion posterior sampling (DPS) algorithms in linear inverse problems. Our test signals are discretized Lévy processes whose posteriors admit efficient Gibbs methods. These Gibbs methods provide gold-standard posterior samples for direct, distribution-level comparisons with DPS algorithms. They can also sample the denoising posteriors in the reverse diffusion, which enables the arbitrary-precision Monte Carlo estimation of various objects that may be needed in the DPS algorithms, such as the expectation or the covariance of the denoising posteriors. In turn, this can be used to isolate algorithmic errors from the errors due to learned components. We instantiate the benchmark with the minimum-mean-squared-error optimality gap and posterior-coverage tests and evaluate popular algorithms on the inverse problems of denoising, deconvolution, imputation, and reconstruction from partial Fourier measurements. We release the benchmark code at https://github.com/emblem-saying/dps-benchmark and invite the community to contribute and report results.

Over the years, computational imaging with accurate nonlinear physical models has drawn considerable interest due to its ability to achieve high-quality reconstructions. However, such nonlinear models are computationally demanding. A popular choice for solving the corresponding inverse problems is accelerated stochastic proximal methods (ASPMs), with the caveat that each iteration is expensive. To overcome this issue, we propose a mini-batch quasi-Newton proximal method (BQNPM) tailored to image-reconstruction problems with total-variation regularization. It involves an efficient approach that computes a weighted proximal mapping at a cost similar to that of the proximal mapping in ASPMs. However, BQNPM requires fewer iterations than ASPMs to converge. We assess the performance of BQNPM on three-dimensional inverse-scattering problems with linear and nonlinear physical models. Our results on simulated and real data show the effectiveness and efficiency of BQNPM,

Geometric misalignments between the nominal and true acquisition parameters in tomography degrade reconstructions. We propose a framework that jointly reconstructs the volume and calibrates the acquisition geometry for arbitrary source–detector configurations. The core of our framework is an x-ray transform operator whose gradients with respect to the acquisition geometry can be efficiently computed with a ray-tracing method of structure and computational complexity similar to those of the forward operator. We represent the volume in a B-spline basis to provide a continuously differentiable model. This results in a better-behaved optimization landscape compared to voxel-based representations. We validate our framework with CT, micro-CT, nano-CT, and positron emission tomography data under a variety of geometric misalignments.

Phase retrieval is a nonlinear inverse problem that arises in a wide range of imaging modalities, from electron microscopy to Fourier ptychography. In particular, the reconstruction is facilitated when the sensing matrix is i.i.d. random, enabling strong theoretical guarantees and efficient reconstruction algorithms. However, its applicability is restricted by excessive computational costs. In this paper, we propose structured random models for phase retrieval, where we emulate a dense random matrix by a cascade of structured transforms and random diagonal matrices. We reduce the complexity from quadratic to log-linear at no cost in reconstruction performance. Through a spectral method initialization followed by gradient descent, robust reconstruction is obtained at an oversampling ratio as low as 2.8. Moreover, we observe that the reconstruction performance is solely determined by the singular-value distribution of the forward matrix. This class of models can directly be implemented with basic optical elements such as lenses and diffusers, paving the way for large-scale phase imaging with robust reconstruction guarantees.

The sampling of functions of bounded variation (BV) is a long-standing problem in op- timization. The ability to sample such functions has relevance in the field of variational inverse problems, where the standard theory fails to guarantee the mere existence of solutions when the loss functional involves samples of BV functions. In this paper, we prove the continuity of sampling functionals and show that the differential operator D admits a unique local inverse. This canonical inversion enables us to formulate an existence theorem for a class of regularized optimization problems that incorporate samples of BV functions. Finally, we characterize the solution set in terms of its extreme points.

We aim at the solution of inverse problems in imaging, by combining a penalized sparse representation of image patches with an unconstrained smooth one. This allows for a straightforward interpretation of the reconstruction. We formulate the optimization as a bilevel problem. The inner problem deploys classical algorithms while the outer problem optimizes the dictionary and the regularizer parameters through supervised learning. The process is carried out via implicit differentiation and gradient-based optimization. We evaluate our method for denoising, super-resolution, and compressed-sensing magnetic-resonance imaging. We compare it to other classical models as well as deep-learning-based methods and show that it always outperforms the former and also the latter in some instances.

Optoacoustic (OA) imaging combined with reversibly photoswitchable proteins has emerged as a promising technology for the high-sensitivity and multiplexed imaging of cells in live tissues in preclinical research. Through carefully designed illumination schedules of ON and OFF laser pulses, the resulting OA signal is a multiplex of different reporter species and the background. We propose a model-based variational framework to computationally unmix and image different species of photo-switching reporters using optoacoustic tomography. It is based on a detailed mathematical description of the photo-switching mechanism, which models how relevant physical parameters such as the kinetic constants and light fluence impact the switching signal. We introduce an algorithm that operates on images, as opposed to traditional pixelwise approaches. It takes the form of an iterative inversion combined with tailored ℓ1 and total-variation regularization to increase the robustness to noise and to improve the unmixing quality. We show that our method can disentangle multiple spatially overlapping labels and recover continuous maps of quantities of interest on controlled phantoms and mice experiments.

Positron emission tomography (PET) scanners use scintillation crystals to stop high-energy photons. The ensuing lower-energy photons are then detected via photomultipliers. We study the performance of a stack of monolithic siliconpixel detectors as an alternative to the combination of crystals and photomultipliers. The resulting design allows for pitches as small as 100 mu m and greatly mitigates depth-of-interaction problems. We develop a theory to optimize the sensitivity of these and other scanners under design constraints. The insight is complemented by Monte Carlo simulations and reconstructions thereof. Experiments and theory alike suggest that our approach has the potential to move PET closer to the microscopic scale. The volumetric resolution is an order of magnitude better than that of the state of the art and the parallax error is very small. A small-animal scanner is now under construction.

We seek to recover an unknown signal from nonlinear amplitude-only measurements, a challenging inverse problem. Strong theoretical guarantees have been established for idealized random measurements, defining the sampling ratio required for signal recovery. However, these results neglect signal priors, which can fundamentally shift these limits, potentially enabling reconstruction with far fewer measurements and simpler models. We evaluate a variety of image priors in the context of severe undersampling with physically-grounded random measurement models. Our results show that these priors enable accurate recovery well below the weak recovery limit, the theoretical threshold required for recovery better than a random guess.

Supervised deep learning has become the method of choice for image denoising. It involves the training of neural networks on large datasets composed of pairs of noisy and clean images. However, the necessity of training data that are specific to the targeted application constrains the widespread use of denoising networks. Recently, several approaches have been developed to overcome this difficulty by whether artificially generating realistic clean/noisy image pairs, or training exclusively on noisy images. In this paper, we show that, contrary to popular belief, denoising networks specialized in the removal of Gaussian noise can be efficiently leveraged in favor of real-world image denoising, even without additional training. For this to happen, an appropriate variance-stabilizing transform (VST) has to be applied beforehand. We propose an algorithm termed Noise2VST for the learning of such a model-free VST. Our approach requires only the input noisy image and an off-the-shelf Gaussian denoiser. We demonstrate through extensive experiments the efficiency and superiority of Noise2VST in comparison to existing methods trained in the absence of specific clean/noisy pairs.

The stiffness of cells and of their nuclei is a biomarker of several pathological conditions. Current measurement methods rely on invasive physical probes that yield one or two stiffness values for the whole cell. However, the internal distribution of cells is heterogeneous. We propose a framework to estimate maps of intracellular and intranuclear stiffness inside deforming cells from fluorescent image sequences. Our scheme requires the resolution of two inverse problems. First, we use a novel optical-flow method that penalizes the nuclear norm of the Hessian to favor deformations that are continuous and piecewise linear, which we show to be compatible with elastic models. We then invert these deformations for the relative intracellular stiffness using a novel system of elliptic PDEs. Our method operates in quasi-static conditions and can still provide relative maps even in the absence of knowledge about the boundary conditions. We compare the accuracy of both methods to the state of the art on simulated data. The application of our method to real data of different cell strains allows us to distinguish different regions inside their nuclei.