Understanding the Influence of Digraphs on Decentralized Optimization: Effective Metrics, Lower Bound, and Optimal Algorithm
arxiv(2023)
摘要
This paper investigates the influence of directed networks on decentralized
stochastic non-convex optimization associated with column-stochastic mixing
matrices. Surprisingly, we find that the canonical spectral gap, a widely used
metric in undirected networks, is insufficient to characterize the impact of
directed topology on decentralized algorithms. To overcome this limitation, we
introduce a novel metric termed equilibrium skewness. This metric, together
with the spectral gap, accurately and comprehensively captures the influence of
column-stochastic mixing matrices on decentralized stochastic algorithms. With
these two metrics, we clarify, for the first time, how the directed network
topology influences the performance of prevalent algorithms such as Push-Sum
and Push-Diging. Furthermore, we establish the first lower bound of the
convergence rate for decentralized stochastic non-convex algorithms over
directed networks. Since existing algorithms cannot match our lower bound, we
further propose the MG-Push-Diging algorithm, which integrates Push-Diging with
a multi-round gossip technique. MG-Push-Diging attains our lower bound up to
logarithmic factors, demonstrating its near-optimal performance and the
tightness of the lower bound. Numerical experiments verify our theoretical
results.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要