Worst-case Evaluation Complexity of a Derivative-Free Quadratic Regularization Method

This manuscript presents a derivative-free quadratic regularization method for unconstrained minimization of a smooth function with Lipschitz continuous gradient. At each iteration, trial points are computed by minimizing a quadratic regularization of a local model of the objective function. The models are based on forward finite-difference gradient approximations. By using a suitable acceptance condition for the trial points, the accuracy of the gradient approximations is dynamically adjusted as a function of the regularization parameter used to control the stepsizes. Worst-case evaluation complexity bounds are established for the new method. Specifically, for nonconvex problems, it is shown that the proposed method needs at most 𝒪( nϵ ^-2) function evaluations to generate an ϵ -approximate stationary point, where n is the problem dimension. For convex problems, an evaluation complexity bound of 𝒪( nϵ ^-1) is obtained, which is reduced to 𝒪( nlog (ϵ ^-1)) under strong convexity. Numerical results illustrating the performance of the proposed method are also reported.