We show that the minimizers of regularized quadratic functions restricted to their natural Krylov spaces increase in Euclidean norm as the spaces expand.
Given a twice-continuously differentiable vector-valued function r(x), a local minimizer of \(\Vert r(x)\Vert _2\) is sought. We propose and analyse tensor-Newton methods, in which r(x) is replaced locally by its second-order Taylor approximation. Convergence is controlled by regularization of various orders. We establish global convergence to a first-order critical point of \(\Vert r(x)\Vert _2\), and provide function evaluation bounds that agree with the best-known bounds for methods using second derivatives. Numerical experiments comparing tensor-Newton methods with regularized Gauss–Newton and Newton methods demonstrate the practical performance of the newly proposed method.
We describe an algorithm for achieving a bordered block triangular form, including the case where the diagonal blocks are themselves in this form. The original algorithm was created by Hellerman and Rarick (1971, 1972) for solving linear programming problems. We include an extension of this algorithm for overcoming the problem of structural singularity in intermediate steps introduced by Erisman, Grimes, Lewis and Poole (1985). Two areas requiring further work include: dealing with safeguards against numerical instability when factorizing the diagonal blocks of the form, and dealing with parallelization, cache management, and memory management for large problems. Only after doing these things would it be appropriate to explore what niche, if any, these algorithms might address when implemented in a modern computer architecture for the solution of very large problems.
