Robustness of Decentralised Learning to Nodes and Data Disruption
arxiv(2024)
摘要
In the vibrant landscape of AI research, decentralised learning is gaining
momentum. Decentralised learning allows individual nodes to keep data locally
where they are generated and to share knowledge extracted from local data among
themselves through an interactive process of collaborative refinement. This
paradigm supports scenarios where data cannot leave local nodes due to privacy
or sovereignty reasons or real-time constraints imposing proximity of models to
locations where inference has to be carried out. The distributed nature of
decentralised learning implies significant new research challenges with respect
to centralised learning. Among them, in this paper, we focus on robustness
issues. Specifically, we study the effect of nodes' disruption on the
collective learning process. Assuming a given percentage of "central" nodes
disappear from the network, we focus on different cases, characterised by (i)
different distributions of data across nodes and (ii) different times when
disruption occurs with respect to the start of the collaborative learning task.
Through these configurations, we are able to show the non-trivial interplay
between the properties of the network connecting nodes, the persistence of
knowledge acquired collectively before disruption or lack thereof, and the
effect of data availability pre- and post-disruption. Our results show that
decentralised learning processes are remarkably robust to network disruption.
As long as even minimum amounts of data remain available somewhere in the
network, the learning process is able to recover from disruptions and achieve
significant classification accuracy. This clearly varies depending on the
remaining connectivity after disruption, but we show that even nodes that
remain completely isolated can retain significant knowledge acquired before the
disruption.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要