A Fast Randomized Incremental Gradient Method for Decentralized Nonconvex Optimization
IEEE Transactions on Automatic Control(2022)
摘要
In this article, we study decentralized nonconvex finite-sum minimization problems described over a network of nodes, where each node possesses a local batch of data samples. In this context, we analyze a single-timescale randomized incremental gradient method, called
GT-SAGA
.
GT-SAGA
is computationally efficient as it evaluates one component gradient per node per iteration and achieves provably fast and robust performance by leveraging node-level variance reduction and network-level gradient tracking. For general smooth nonconvex problems, we show the almost sure and mean-squared convergence of
GT-SAGA
to a first-order stationary point and further describe regimes of practical significance, where it outperforms the existing approaches and achieves a network topology-independent iteration complexity, respectively. When the global function satisfies the Polyak–Łojaciewisz condition, we show that
GT-SAGA
exhibits linear convergence to an optimal solution in expectation and describe regimes of practical interest where the performance is network topology independent and improves upon the existing methods. Numerical experiments are included to highlight the main convergence aspects of
GT-SAGA
in nonconvex settings.
更多查看译文
关键词
Decentralized nonconvex optimization,incremental gradient methods,variance reduction
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要