Accelerated Variance-Reduced Random Reshuffling Gradient Descent Algorithm for Nonconvex Finite-Sum Optimization

Although stochastic gradient descent (SGD) is well-established theoretically, most widely used machine learning implementations employ sampling without replacement because of its empirical superiority. Specifically, training data are randomly shuffled (or permuted) before each epoch, a strategy known as random reshuffling (RR), and training proceeds sequentially according to the shuffled order. In this paper, we propose an accelerated variance-reduced random reshuffling gradient descent algorithm (AVR-RRGD) for nonconvex finite-sum optimization problems. Although the convergence and acceleration effects of random reshuffling methods combined with variance reduction are well studied in the convex setting, the nonconvex case remains significantly less explored. Under mild assumptions, we first establish weak convergence under appropriate stepsize conditions. Leveraging the Kurdyka-& Lstrok;ojasiewicz (KL) inequality, we then prove strong limit-point convergence and derive convergence rates dependent on the KL exponent and stepsize choices. Finally, experiments on ridge regression and logistic regression confirm the efficiency of our approach.