On the Powerball Method: Variants of Descent Methods for Accelerated Optimization

We propose a new method to accelerate the convergence of optimization algorithms. This method, termed Powerball method, simply adds a power coefficient γ ∈ [0,1) to the gradient used in various gradient-based optimization schemes. In its essence, the Powerball method can be regarded as the steepest gradient descent with respect to the p-norm, where p=1+(1/γ). As a motivation, we first present the ...