AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
We develop consensus-based distributed algorithms and establish the regret bounds for both convex and strongly convex losses, which match those of the centralized online optimization in the literature

Online Convex Optimization Over Erdos-Renyi Random Networks

NIPS 2020, (2020)

被引用0|浏览43
EI
下载 PDF 全文
引用
微博一下

摘要

The work studies how node-to-node communications over an Erdos-Rényi random network influence distributed online convex optimization, which is vital in solving large-scale machine learning in antagonistic or changing environments. At per step, each node (computing unit) makes a local decision, experiences a loss evaluated with a convex fu...更多

代码

数据

0
简介
  • The online convex optimization paradigm has become a central and canonical solution for machine learning where data is generated sequentially over time, e.g., online routing, ad. selection for search engines, and spam filtering ([1,2,3,4]).
  • The goal of the learner is to minimize its regret by adapting the decisions along the streaming data, measured by the difference between the cumulative loss of online decisions and the loss of the best decision chosen in hindsight.
  • The gradient descent algorithm was proved to guarantee regret bounds O( T ) and O(ln(T )) for convex and strongly convex loss functions ([5, 6]), respectively, which were later shown to be minimax optimal ([7, 8])
重点内容
  • The online convex optimization paradigm has become a central and canonical solution for machine learning where data is generated sequentially over time, e.g., online routing, ad. selection for search engines, and spam filtering ([1,2,3,4])
  • When the nodes can only observe loss function values, the one-point bandit or two-points bandit around the current decision is used to get a randomized approximation of the gradient. √
  • We develop consensus-based distributed algorithms and establish the regret bounds for both convex and strongly convex losses, which match those of the centralized online optimization in the literature
  • We further quantitatively characterize the influence of the algebraic network connectivity on the regret bounds, and show that the link connection probability can be used to tune a trade-off between the communication overhead and the computation accuracy
  • Future directions include closing the gap of the regret bounds and extending the kernel-based methods to bandit online convex optimization over networks
  • The work provides the theoretical understanding of the performance limits about distributed online convex optimization over random networks, and could be applied in processing streaming data to various Internet of Things systems, such as machine learning with personal wearable devices
结果
  • The authors consider online distributed gradient descents under Erdos-Rényi graphs, and establish the regret bounds explicitly in term of time horizon T , the underlying graph G, the probability p, and the decision complexity d.
  • Algorithms: The node adapts its decision with gradient descents and local averaging over Erdos-.
  • When the nodes can only observe loss function values, the one-point bandit or two-points bandit around the current decision is used to get a randomized approximation of the gradient.
  • It is shown that the regrets scale with network size by a magnitude of
结论
  • The authors consider the online convex optimization over Erdos-Rényi random graphs under the full information feedback, one-point and two-points bandit feedback.
  • The work provides the theoretical understanding of the performance limits about distributed online convex optimization over random networks, and could be applied in processing streaming data to various Internet of Things systems, such as machine learning with personal wearable devices.
  • It does not present any foreseeable societal consequence
表格
  • Table1: Regret bounds and communication complexity over classical Erdos-Rényi graphs
Download tables as Excel
相关工作
  • The early works [5] and [6] a√bout the centralized online convex optimization in the full information feedback obtained regrets O( T ) and O(ln(T )) for convex and strongly convex losses, respectively. With one-point bandit feedback, the seminal work [15] modified the gradient descent algorithm by replacing the gradient with a randomized estimate, and showed that the expected regret was O(T 3/4) for bounded and Lipschitz-continuous convex losses, whereas the regret O(T 2/3) was obtained in [16] for the setting of Lipschitz and strongly convex losses. It remains an open problem to design an optimal algorithm for the one-point band√it online convex optimization, whereas [17] proved that the optimal regret can not be better than Ω( T ) e√ven for strongly convex losses. In some special cases, the minimax regret is exactly O(poly(ln(T )) T ) , e.g., the losses are Lipschitz and linear [17, 18], or they are both smooth and str√ongly-convex [19]. The recent work [20] designed some kernel-based methods with O(poly(ln(T )) T ) regret and polynomial computing time. [16] extended the onepoint bandit feedback to the multi-points bandit feedback where loss can b√e observed at multiple points around the decision, and established the expected regret bounds O( T ) and O(ln(T )) for convex and strongly convex losses. In this work, we design distributed algorithms over random graphs with full gradient feedback, one-point bandit feedback, and two-points bandit feedback, which can recover the regret bounds in the centralized methods ([6], [15] and [16]).
基金
  • The work was sponsored by Shanghai Sailing Program (No 20YF1453000, No 20YF1452800) and the Fundamental Research Funds for the Central Universities, China (No 22120200047, No 22120200048)
引用论文
  • Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends R in Optimization, 2(3-4):157–325, 2016.
    Google ScholarLocate open access versionFindings
  • Ashok Cutkosky and Kwabena A Boahen. Online convex optimization with unconstrained domains and losses. In Advances in Neural Information Processing Systems (NIPS), pages 748–756, 2016.
    Google ScholarLocate open access versionFindings
  • Jianjun Yuan and Andrew Lamperski. Online convex optimization for cumulative constraints. In Advances in Neural Information Processing Systems (NIPS), pages 6137–6146, 2018.
    Google ScholarLocate open access versionFindings
  • Sébastien Arnold, Pierre-Antoine Manzagol, Reza Babanezhad Harikandeh, Ioannis Mitliagkas, and Nicolas Le Roux. Reducing the variance in online optimization by transporting past gradients. In Advances in Neural Information Processing Systems (NIPS), pages 5392–5403, 2019.
    Google ScholarLocate open access versionFindings
  • Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (ICML-03), pages 928–936, 2003.
    Google ScholarLocate open access versionFindings
  • Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex optimization. Machine Learning, 69(2-3):169–192, 2007.
    Google ScholarLocate open access versionFindings
  • Jacob Abernethy, Alekh Agarwal, and Peter L Bartlett. A stochastic view of optimal regret through minimax duality. In Proceedings of the 22nd Annual Conference on Learning Theory (COLT), 2009.
    Google ScholarLocate open access versionFindings
  • Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cambridge university press, 2006.
    Google ScholarFindings
  • Wenpeng Zhang, Peilin Zhao, Wenwu Zhu, Steven CH Hoi, and Tong Zhang. Projection-free distributed online learning in networks. In Proceedings of the 34th International Conference on Machine LearningVolume 70 (ICML), pages 4054–4062. JMLR. org, 2017.
    Google ScholarLocate open access versionFindings
  • Nuwan Ferdinand, Haider Al-Lawati, Stark Draper, and Matthew Nokleby. Anytime minibatch: Exploiting stragglers in online distributed optimization. In Seventh International Conference on Learning Representations (ICLR), 2019.
    Google ScholarLocate open access versionFindings
  • Paul Erdos and Alfréd Rényi. On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci, 5(1): 17–60, 1960.
    Google ScholarLocate open access versionFindings
  • Osman Yagan and Armand M Makowski. Modeling the pairwise key predistribution scheme in the presence of unreliable links. IEEE Transactions on Information Theory, 59(3):1740–1760, 2012.
    Google ScholarLocate open access versionFindings
  • Osman Yagan. Performance of the eschenauer–gligor key distribution scheme under an on/off channel. IEEE Transactions on Information Theory, 58(6):3821–3835, 2012.
    Google ScholarLocate open access versionFindings
  • Dhaval Adjodah, Dan Calacci, Abhimanyu Dubey, Anirudh Goyal, Peter Krafft, Esteban Moro, and Alex Pentland. Communication topologies between learning agents in deep reinforcement learning. arXiv preprint arXiv:1902.06740, 2019.
    Findings
  • Abraham D Flaxman, Adam Tauman Kalai, Adam Tauman Kalai, and H Brendan McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. In Proceedings of the sixteenth annual ACM-SIAM symposium on Discrete algorithms, pages 385–394. Society for Industrial and Applied Mathematics, 2005.
    Google ScholarLocate open access versionFindings
  • Alekh Agarwal, Ofer Dekel, and Lin Xiao. Optimal algorithms for online convex optimization with multi-point bandit feedback. In Proceedings of the 23rd Annual Conference on Learning Theory (COLT), pages 28–40, 2010.
    Google ScholarLocate open access versionFindings
  • Varsha Dani, Sham M Kakade, and Thomas P Hayes. The price of bandit information for online optimization. In Advances in Neural Information Processing Systems (NIPS), pages 345–352, 2008.
    Google ScholarLocate open access versionFindings
  • Jacob Abernethy, Elad E Hazan, and Alexander Rakhlin. Competing in the dark: An efficient algorithm for bandit linear optimization. In 21st Annual Conference on Learning Theory, COLT 2008, pages 263–273, 2008.
    Google ScholarLocate open access versionFindings
  • Elad Hazan and Kfir Levy. Bandit convex optimization: Towards tight bounds. In Advances in Neural Information Processing Systems, pages 784–792, 2014.
    Google ScholarLocate open access versionFindings
  • Sébastien Bubeck, Yin Tat Lee, and Ronen Eldan. Kernel-based methods for bandit convex optimization. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pages 72–85, 2017.
    Google ScholarLocate open access versionFindings
  • Deming Yuan, Alexandre Proutiere, and Guodong Shi. Distributed online linear regression. IEEE Transactions on Information Theory, in press, Preprint arXiv:1902.04774, 2019.
    Findings
  • Feng Yan, Shreyas Sundaram, SVN Vishwanathan, and Yuan Qi. Distributed autonomous online learning: Regrets and intrinsic privacy-preserving properties. IEEE Transactions on Knowledge and Data Engineering, 25(11):2483–2493, 2012.
    Google ScholarLocate open access versionFindings
  • Saghar Hosseini, Airlie Chapman, and Mehran Mesbahi. Online distributed optimization via dual averaging. In 52nd IEEE Conference on Decision and Control, pages 1484–1489. IEEE, 2013.
    Google ScholarLocate open access versionFindings
  • David Mateos-Nunez and Jorge Cortés. Distributed online convex optimization over jointly connected digraphs. IEEE Transactions on Network Science and Engineering, 1(1):23–37, 2014.
    Google ScholarLocate open access versionFindings
  • Deming Yuan, Alexandre Proutiere, and Guodong Shi. Distributed online optimization with long-term constraints. Preprint arXiv:1912.09705, 2019.
    Findings
  • Mohammad Akbari, Bahman Gharesifard, and Tamás Linder. Distributed online convex optimization on time-varying directed graphs. IEEE Transactions on Control of Network Systems, 4(3):417–428, 2015.
    Google ScholarLocate open access versionFindings
  • Soomin Lee and Michael M Zavlanos. Distributed primal-dual methods for online constrained optimization. In 2016 American Control Conference (ACC), pages 7171–7176. IEEE, 2016.
    Google ScholarLocate open access versionFindings
  • Elad Hazan, Alexander Rakhlin, and Peter L Bartlett. Adaptive online gradient descent. In Advances in Neural Information Processing Systems (NIPS), pages 65–72, 2008.
    Google ScholarLocate open access versionFindings
  • Fabio Fagnani and Sandro Zampieri. Randomized consensus algorithms over large scale networks. IEEE Journal on Selected Areas in Communications, 26(4):634–649, 2008.
    Google ScholarLocate open access versionFindings
  • Theodora S Brisimi, Ruidi Chen, Theofanie Mela, Alex Olshevsky, Ioannis Ch Paschalidis, and Wei Shi. Federated learning of predictive models from federated electronic health records. International journal of medical informatics, 112:59–67, 2018.
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科