A New Perspective of Accelerated Gradient Methods: the Controlled Invariant Manifold Approach

Gradient Descent (GD) is a ubiquitous algorithm for finding the optimal solution to an optimization problem. For reduced computational complexity, the optimal solution x^* of the optimization problem must be attained in a minimum number of iterations. For this objective, the paper proposes a genesis of an accelerated gradient algorithm through the controlled dynamical system perspective. The objective of optimally reaching the optimal solution x^* where ∇ f(x^*)=0 with a given initial condition x(0) is achieved through control.