Learning Hyperbolic Representations of Topological Features

Panagiotis Kyriakis
Panagiotis Kyriakis
Iordanis Fostiropoulos
Iordanis Fostiropoulos

international conference on learning representations, 2020.

Cited by: 0|Views10
Weibo:
We develop a method to learn representations of topological features (i.e., persistence diagrams) on hyperbolic spaces.

Abstract:

Learning task-specific representations of persistence diagrams is an important problem in topological data analysis and machine learning. However, current state of the art methods are restricted in terms of their expressivity as they are focused on Euclidean representations. Persistence diagrams often contain features of infinite persiste...More

Code:

Data:

0
Full Text
Bibtex
Weibo
Introduction
  • Persistent homology is a topological data analysis tool which tracks how topological features appear and disappear as the authors analyze the data at different scales or in nested sequences of subspaces [1; 2].
  • Under some assumptions, one can approximately reconstruct the input space from a diagram [4]
  • Despite their strengths, the space of persistence diagrams lacks structure as basic operations, such as addition and scalar multiplication, are not defined.
  • A filtration of K is a nested sequence of subcomplexes that starts with the empty complex and ends with K,
Highlights
  • Persistent homology is a topological data analysis tool which tracks how topological features appear and disappear as we analyze the data at different scales or in nested sequences of subspaces [1; 2]
  • The lifespan of the feature is called persistence. Persistent homology summarizes these topological characteristics in a form of multiset called persistence diagram, which are highly robust and versatile descriptors of the data
  • We focus on persistence diagrams extracted from graphs and grey-scale images
  • To the best of our knowledge, method for learning representations of persistence diagrams in the Poincare ball
  • The main benefit of using the Poincare space is that by allowing the representations of essential features to get infinitesimally close to the boundary of the ball their distance to non-essential features approaches infinity, preserves their relative importance
  • Directions for future work include the learning of filtration and/or scale end-to-end as well as to investigate whether or not there exists some hyperbolic trend in distances appearing in persistence diagrams that justify the improved performance especially on small graphs
Methods
  • The authors focus on persistence diagrams extracted from graphs and grey-scale images.
  • In both cases, the learning task is classification and the authors compare the performance of the method against other methods.
  • The authors' representation acts as an input to a neural network architecture and the parameters are learned end-to-end via standard gradient methods.
  • The authors implemented all algorithms in TensorFlow 2.2 using the TDA-Toolkit and the Scikit-TDA3 for extracting persistence diagrams and run all experiments on the Google Cloud AI Platform.
  • The authors provide the code to reproduce the results in the supplementary material
Results
  • Let D, E be two persistence diagrams and η : D → E bijection that achieves the infinum in Eq..
  • Consider the subset D0 = D \ R∆, which is essentially the original diagram without the points in the diagonal.
  • The authors have the following sequence of inequalities dB(Φ(D, θ), Φ(E, θ)) [21] = dB expx0.
  • Logx0 φ(ρ(x)) , expx0 logx0 φ(ρ(x)) [22] x∈E.
  • ≤ Ke logx0 (φ(ρ(x))) − logx0 (φ(ρ(x)))) [23]
Conclusion
  • To the best of the knowledge, method for learning representations of persistence diagrams in the Poincare ball.
  • The authors' main motivation for introducing such method is that persistence diagrams often contain topological features of infinite persistence the representational capacity of which may be bottlenecked when representing them in Euclidean spaces.
  • This stems from the fact that Euclidean spaces cannot assign infinite distance to finite points.
  • Directions for future work include the learning of filtration and/or scale end-to-end as well as to investigate whether or not there exists some hyperbolic trend in distances appearing in persistence diagrams that justify the improved performance especially on small graphs
Summary
  • Introduction:

    Persistent homology is a topological data analysis tool which tracks how topological features appear and disappear as the authors analyze the data at different scales or in nested sequences of subspaces [1; 2].
  • Under some assumptions, one can approximately reconstruct the input space from a diagram [4]
  • Despite their strengths, the space of persistence diagrams lacks structure as basic operations, such as addition and scalar multiplication, are not defined.
  • A filtration of K is a nested sequence of subcomplexes that starts with the empty complex and ends with K,
  • Methods:

    The authors focus on persistence diagrams extracted from graphs and grey-scale images.
  • In both cases, the learning task is classification and the authors compare the performance of the method against other methods.
  • The authors' representation acts as an input to a neural network architecture and the parameters are learned end-to-end via standard gradient methods.
  • The authors implemented all algorithms in TensorFlow 2.2 using the TDA-Toolkit and the Scikit-TDA3 for extracting persistence diagrams and run all experiments on the Google Cloud AI Platform.
  • The authors provide the code to reproduce the results in the supplementary material
  • Results:

    Let D, E be two persistence diagrams and η : D → E bijection that achieves the infinum in Eq..
  • Consider the subset D0 = D \ R∆, which is essentially the original diagram without the points in the diagonal.
  • The authors have the following sequence of inequalities dB(Φ(D, θ), Φ(E, θ)) [21] = dB expx0.
  • Logx0 φ(ρ(x)) , expx0 logx0 φ(ρ(x)) [22] x∈E.
  • ≤ Ke logx0 (φ(ρ(x))) − logx0 (φ(ρ(x)))) [23]
  • Conclusion:

    To the best of the knowledge, method for learning representations of persistence diagrams in the Poincare ball.
  • The authors' main motivation for introducing such method is that persistence diagrams often contain topological features of infinite persistence the representational capacity of which may be bottlenecked when representing them in Euclidean spaces.
  • This stems from the fact that Euclidean spaces cannot assign infinite distance to finite points.
  • Directions for future work include the learning of filtration and/or scale end-to-end as well as to investigate whether or not there exists some hyperbolic trend in distances appearing in persistence diagrams that justify the improved performance especially on small graphs
Tables
  • Table1: Classification accuracy (mean±std or min-max range, if available)
  • Table2: Classification accuracy
Download tables as Excel
Related work
  • To address these issues, several vectorization methods have been proposed. Some of the earliest approaches are based on kernels, i.e., generalized products that turn persistence diagrams into elements of a Hilbert space. Kusano et al [5] propose a persistence weighted Gaussian kernel which allows them to explicitly control the effect of persistence. Alternatively, Carrière et al [6] leverage the sliced Wasserstein distance to define a kernel that mimics the distance between diagrams. The approaches by Bubenik [7] based on persistent landscapes, by Reininghaus et al [8] based on scale space theory and by Le et al [9] based on the Fisher information metric are along the same line of work. The major drawback in utilizing kernel methods is that they suffer from scalability issues as the training scales poorly with the number of samples.
Study subjects and analysis
samples: 1000
We evaluate our approach using social graphs from [26]. The REDDIT-BINARY dataset contains 1000 samples and the graphs correspond to online discussion threads. The task is to identify to which community a given graph belongs (question/answer-based community or a discussion-based community)

Reference
  • Herbert Edelsbrunner and John Harer. Persistent homology—a survey. Discrete Computational Geometry - DCG, 453, 01 2008.
    Google ScholarLocate open access versionFindings
  • Herbert Edelsbrunner and D. Morozov. Persistent homology: theory and practice. European Congress of Mathematics, pages 31–50, 01 2014.
    Google ScholarLocate open access versionFindings
  • David Cohen-Steiner, Herbert Edelsbrunner, and John Harer. Stability of persistence diagrams. Discrete & Computational Geometry, 37(1):103–120, 2007.
    Google ScholarLocate open access versionFindings
  • K. Turner, S. Mukherjee, and D. M. Boyer. Persistent homology transform for modeling shapes and surfaces. Information and Inference: A Journal of the IMA, 3(4):310–344, Dec 2014.
    Google ScholarLocate open access versionFindings
  • Genki Kusano, Kenji Fukumizu, and Yasuaki Hiraoka. Persistence weighted gaussian kernel for topological data analysis. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, page 2004–2013. JMLR.org, 2016.
    Google ScholarLocate open access versionFindings
  • Mathieu Carrière, Marco Cuturi, and Steve Oudot. Sliced wasserstein kernel for persistence diagrams. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 664–673. JMLR.org, 2017.
    Google ScholarLocate open access versionFindings
  • Peter Bubenik. Statistical topological data analysis using persistence landscapes. Journal of Machine Learning Research, 16(3):77–102, 2015.
    Google ScholarLocate open access versionFindings
  • J. Reininghaus, S. Huber, U. Bauer, and R. Kwitt. A stable multi-scale kernel for topological machine learning. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4741–4748, 2015.
    Google ScholarLocate open access versionFindings
  • Tam Le and Makoto Yamada. Persistence fisher kernel: A riemannian manifold kernel for persistence diagrams. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 10007– 10018. Curran Associates, Inc., 2018.
    Google ScholarLocate open access versionFindings
  • Henry Adams, Tegan Emerson, Michael Kirby, Rachel Neville, Chris Peterson, Patrick Shipman, Sofya Chepushtanova, Eric Hanson, Francis Motta, and Lori Ziegelmeier. Persistence images: A stable vector representation of persistent homology. Journal of Machine Learning Research, 18(8):1–35, 2017.
    Google ScholarLocate open access versionFindings
  • Frédéric Chazal, Brittany Terese Fasy, Fabrizio Lecci, Alessandro Rinaldo, and Larry Wasserman. Stochastic convergence of persistence landscapes and silhouettes. In Proceedings of the Thirtieth Annual Symposium on Computational Geometry, SOCG’14, page 474–483, New York, NY, USA, 2014. Association for Computing Machinery.
    Google ScholarLocate open access versionFindings
  • Barbara Di Fabio and Massimo Ferri. Comparing persistence diagrams through complex vectors. In Vittorio Murino and Enrico Puppo, editors, Image Analysis and Processing — ICIAP 2015, pages 294–305, Cham, 2015. Springer International Publishing.
    Google ScholarLocate open access versionFindings
  • Aaron Adcock, Erik Carlsson, and Gunnar Carlsson. The ring of algebraic functions on persistence bar codes. Homology, Homotopy and Applications, 18, 04 2013.
    Google ScholarLocate open access versionFindings
  • Sara Kalisnik. Tropical coordinates on the space of persistence barcodes. Foundations of Computational Mathematics, 19(1):101–129, 2019.
    Google ScholarLocate open access versionFindings
  • Christoph Hofer, Roland Kwitt, Marc Niethammer, and Andreas Uhl. Deep learning with topological signatures. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 1633–1643, Red Hook, NY, USA, 2017. Curran Associates Inc.
    Google ScholarLocate open access versionFindings
  • Christoph D. Hofer, Roland Kwitt, and Marc Niethammer. Learning representations of persistence barcodes. Journal of Machine Learning Research, 20(126):1–45, 2019.
    Google ScholarLocate open access versionFindings
  • Mathieu Carrière, Frédéric Chazal, Yuichi Ike, Théo Lacombe, Martin Royer, and Yuhei Umeda. Perslay: A neural network layer for persistence diagrams and new graph topological signatures. In Silvia Chiappa and Roberto Calandra, editors, The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy], volume 108 of Proceedings of Machine Learning Research, pages 2786–2796. PMLR, 2020.
    Google ScholarLocate open access versionFindings
  • M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst. Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, 2017.
    Google ScholarLocate open access versionFindings
  • Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 6338–6347. Curran Associates, Inc., 2017.
    Google ScholarLocate open access versionFindings
  • Maximilian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 3776–3785. PMLR, 2018.
    Google ScholarLocate open access versionFindings
  • Frederic Sala, Chris De Sa, Albert Gu, and Christopher Re. Representation tradeoffs for hyperbolic embeddings. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4460–4469, Stockholmsmässan, Stockholm Sweden, 10–15 Jul 2018. PMLR.
    Google ScholarLocate open access versionFindings
  • Octavian Ganea, Gary Becigneul, and Thomas Hofmann. Hyperbolic neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 5345–5355. Curran Associates, Inc., 2018.
    Google ScholarLocate open access versionFindings
  • Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas. Hyperbolic attention networks. In International Conference on Learning Representations, 2019.
    Google ScholarLocate open access versionFindings
  • Qi Liu, Maximilian Nickel, and Douwe Kiela. Hyperbolic graph neural networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8230–8241. Curran Associates, Inc., 2019.
    Google ScholarLocate open access versionFindings
  • Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 4868–4879. Curran Associates, Inc., 2019.
    Google ScholarLocate open access versionFindings
  • Pinar Yanardag and S.V.N. Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, page 1365–1374, New York, NY, USA, 2015. Association for Computing Machinery.
    Google ScholarLocate open access versionFindings
  • Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M. Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(77):2539–2561, 2011.
    Google ScholarLocate open access versionFindings
  • Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, page 2014–2023. JMLR.org, 2016.
    Google ScholarLocate open access versionFindings
  • Qi Zhao and Yusu Wang. Learning metrics for persistence-based summaries and applications for graph classification. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada, pages 9855–9866, 2019.
    Google ScholarLocate open access versionFindings
  • Sergey Ivanov and Evgeny Burnaev. Anonymous walk embeddings. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2186–2195, Stockholmsmässan, Stockholm Sweden, 10–15 Jul 2018. PMLR.
    Google ScholarLocate open access versionFindings
  • Christoph Hofer, Florian Graf, Bastian Rieck, Marc Niethammer, and Roland Kwitt. Graph filtration learning. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 4314–4323, Virtual, 13–18 Jul 2020. PMLR.
    Google ScholarLocate open access versionFindings
  • Manfredo Perdigao do Carmo. Riemannian geometry. Birkhäuser, 1992.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments