In the framework of sparsity-enforcing regularisation for linear inverse problems, we consider the minimisation of a square-root Lasso cost function. To solve this problem we devise a simple modification of the iterative soft-thresholding algorithm (ISTA) for the Lasso problem and we prove convergence for this algorithm, which we call SQRT-ISTA. Under some additional assumptions, we derive an upper bound on the convergence rate of the cost function. We also generalise these results to the case of the group square-root Lasso, where sparsity is enforced for groups of variables instead of individual ones.

We present a numerical iterative optimization algorithm for the minimization of a cost function consisting of a linear combination of three convex terms, one of which is differentiable, a second one is prox-simple and the third one is the composition of a linear map and a prox-simple function. The algorithm's special feature lies in its ability to approximate, in a single iteration run, the minimizers of the cost function for many different values of the parameters determining the relative weight of the three terms in the cost function. A proof of convergence of the algorithm, based on an inexact variable metric approach, is also provided. As a special case, one recovers a generalization of the primal-dual algorithm of Chambolle and Pock, and also of the proximal-gradient algorithm. Finally, we show how it is related to a primal-dual iterative algorithm based on inexact proximal evaluations of the non-smooth terms of the cost function.

We consider a variation of the classical proximal-gradient algorithm for the iterative minimization of a cost function consisting of a sum of two terms, one smooth and the other prox-simple, and whose relative weight is determined by a penalty parameter. This so-called fixed-point continuation method allows one to approximate the problem's trade-off curve, i.e. to compute the minimizers of the cost function for a whole range of values of the penalty parameter at once. The algorithm is shown to converge, and a rate of convergence of the cost function is also derived. Furthermore, it is shown that this method is related to iterative algorithms constructed on the basis of the ϵ-subdifferential of the prox-simple term. Some numerical examples are provided.

The convergence of many proximal algorithms involving a gradient descent relies on its Lipschitz constant. To avoid computing it, backtracking rules can be used. While such a rule has already been designed for the forward-backward algorithm (FBwB), this scheme is not flexible enough when a non-differentiable penalization with a linear operator is added to a constraint. In this work we propose a backtracking rule for the primal-dual scheme (PDwB), and evaluate its performance for the epigraphical constrained high dynamical reconstruction in high contrast polarimetric imaging, under TV penalization.

The importance of an adequate inner loop starting point (as opposed to a sufficient inner loop stopping rule) is discussed in the context of a numerical optimization algorithm consisting of nested primal-dual proximal-gradient iterations. While the number of inner iterations is fixed in advance, convergence of the whole algorithm is still guaranteed by virtue of a warm-start strategy for the inner loop, showing that inner loop "starting rules" can be just as effective as "stopping rules" for guaranteeing convergence. The algorithm itself is applicable to the numerical solution of convex optimization problems defined by the sum of a differentiable term and two possibly non-differentiable terms. One of the latter terms should take the form of the composition of a linear map and a proximable function, while the differentiable term needs an accessible gradient. The algorithm reduces to the classical proximal gradient algorithm in certain special cases and it also generalizes other existing algorithms. In addition, under some conditions of strong convexity, we show a linear rate of convergence.

We consider a variable metric linesearch based proximal gradient method for the minimization of the sum of a smooth, possibly nonconvex function plus a convex, possibly nonsmooth term. We prove convergence of this iterative algorithm to a critical point if the objective function satisfies the Kurdyka-Lojasiewicz property at each point of its domain, under the assumption that a limit point exists. The proposed method is applied to a wide collection of image processing problems and our numerical tests show that our algorithm results to be flexible, robust and competitive when compared to recently proposed approaches able to address the optimization problems arising in the considered applications.

The scaled gradient projection (SGP) method is a first-order optimization method applicable to the constrained minimization of smooth functions and exploiting a scaling matrix multiplying the gradient and a variable steplength parameter to improve the convergence of the scheme. For a general nonconvex function, the limit points of the sequence generated by SGP have been proved to be stationary, while in the convex case and with some restrictions on the choice of the scaling matrix the sequence itself converges to a constrained minimum point. In this paper we extend these convergence results by showing that the SGP sequence converges to a limit point provided that the objective function satisfies the Kurdyka-Łojasiewicz property at each point of its domain and its gradient is Lipschitz continuous.

We consider a variable metric line-search based proximal gradient method for the minimization of the sum of a smooth, possibly nonconvex function plus a convex, possibly nonsmooth term. The general convergence result on this method is the stationarity of any limit point of the sequence generated by the method, while convergence of the sequence itself to a minimum point has been proved recently for convex objective functions under some additional hypotheses. In this paper we show that the same convergence result can be proved if the objective function satisfies the Kurdyka-Lojasiewicz property at each point of its domain, under the assumption that a limit point exists. The proposed method is applied to a wide collection of image processing problems and our numerical tests show that our algorithm results to be flexible, robust and competitive if compared to recently proposed approaches able to address the optimization problems arising in the considered applications.

We develop a new proximal-gradient method for minimizing the sum of a differentiable, possibly nonconvex,function plus a convex, possibly nondifferentiable, function. The key features of the proposed method are thedefinition of a suitable descent direction, based on the proximal operator associated to the convex part ofthe objective function, and an Armijo-like rule to determine the stepsize along this direction ensuring thesufficient decrease of the objective function. In this frame, we especially address the possibility ofadopting a metric which may change at each iteration and an inexact computation of the proximal point definingthe descent direction. For the more general nonconvex case, we prove that all limit points of the iteratessequence are stationary, while for convex objective functions we prove the convergence of the whole sequenceto a minimizer, under the assumption that a minimizer exists. In the latter case, assuming also that thegradient of the smooth part of the objective function is Lipschitz, we also give a convergence rate estimate,showing the ${\mathcal O}(\frac 1 k)$ complexity with respect to the function values. We also discussverifiable sufficient conditions for the inexact proximal point and present the results of two numericaltests on total-variation-based image restoration problems, showing that the proposed approach is competitivewith other state-of-the-art methods.

We present a variable metric line--search based proximal--gradient method for the minimization of the sum of a smooth, possibly nonconvex function plus a convex, possibly nonsmooth term. The strong convergence of the method can be proved if the objective function satisfies the Kurdyka-Lojasiewicz property at each point of its domain. Numerical experience on some nonconvex image reconstruction problems shows the proposed approach is competitive with other state-of-the-art methods.

We discuss a number of novel steplength selection schemes for proximal-based convex optimization algorithms. In particular, we consider the problem where the Lipschitz constant of the gradient of the smooth part of the objective function is unknown. We generalize two optimization algorithms of Khobotov type and prove convergence. We also take into account possible inaccurate computation of the proximal operator of the non-smooth part of the objective function. Secondly, we show convergence of an iterative algorithm with Armijo-type steplength rule, and discuss its use with an approximate computation of the proximal operator. Numerical experiments show the efficiency of the methods in comparison to some existing schemes.

Quantile regression is studied in combination with a penalty which promotes structured (or group) sparsity. A mixed $\ell_{1,\infty}$-norm on the parameter vector is used to impose structured sparsity on the traditional quantile regression problem. An algorithm is derived to calculate the piece-wise linear solution path of the corresponding minimization problem. A Matlab implementation of the proposed algorithm is provided and some applications of the methods are also studied.

Pseudo-random numbers are often used for generating incoherent uniformly distributed sample distributions. However randomness is a sufficient – not necessary – condition to ensure incoherence. If one wants to reconstruct an image from few samples, choosing a globally optimized set of evenly distributed points could capture the visual content more efficiently. This work compares classical random sampling with a simple construction based on properties of the fractional Golden ratio sequence and the Hilbert space filling curve. Images are then reconstructed using a total variation prior. Results show improvements in terms of peak signal to noise ratio over pseudo-random sampling.

We propose an iterative algorithm for the minimization of a ℓ 1-norm penalized least squares functional, under additional linear constraints. The algorithm is fully explicit: it uses only matrix multiplications with the three matrices present in the problem (in the linear constraint, in the data misfit part and in the penalty term of the functional). None of the three matrices must be invertible. Convergence is proven in a finite-dimensional setting. We apply the algorithm to a synthetic problem in magneto-encephalography where it is used for the reconstruction of divergence-free current densities subject to a sparsity promoting penalty on the wavelet coefficients of the current densities. We discuss the effects of imposing zero divergence and of imposing joint sparsity (of the vector components of the current density) on the current density reconstruction.

We present a realistic application of an inversion scheme for global seismic tomography that uses as prior information the sparsity of a solution, defined as having few nonzero coefficients under the action of a linear transformation. In this paper, the sparsifying transform is a wavelet transform. We use an accelerated iterative soft‐thresholding algorithm for a regularization strategy, which produces sparse models in the wavelet domain. The approach and scheme we present may be of use for preserving sharp edges in a tomographic reconstruction and minimizing the number of features in the solution warranted by the data. The method is tested on a data set of time delays for finite‐frequency tomography using the USArray network, the first application in global seismic tomography to real data. The approach presented should also be suitable for other imaging problems. From a comparison with a more traditional inversion using damping and smoothing constraints, we show that (1) we generally retrieve similar features, (2) fewer nonzero coefficients under a properly chosen representation (such as wavelets) are needed to explain the data at the same level of root‐mean‐square misfit, (3) the model is sparse or compressible in the wavelet domain, and (4) we do not need to construct a heterogeneous mesh to capture the available resolution.

A new class of probability distributions, the so-called connected double truncated gamma distribution, is introduced. We show that using this class as the error distribution of a linear model leads to a generalized quantile regression model that combines desirable properties of both least-squares and quantile regression methods: robustness to outliers and differentiable loss function.

This article reviews Wavelets: A Concise Guide. by Amir-Homayoon Najmi 270 pp. , Baltimore, 2012. Price: $45.00 (paper) ISBN 978-1-4214-0496-6.