p-Norm Flow Diffusion for Local Graph Clustering

international conference on machine learning, 2020.

Cited by: 0|Bibtex|Views5
Other Links: academic.microsoft.com
Keywords:
local structurediffusion methodlocal clusteringspectral methodoptimization formulationMore(5+)
Weibo:
Local algorithms for local graph clustering are predominantly based on the idea of diffusion, which is the generic process of spreading mass among vertices by sending mass along edges

Abstract:

Local graph clustering and the closely related seed set expansion problem are primitives on graphs that are central to a wide range of analytic and learning tasks such as local clustering, community detection, nodes ranking and feature inference. Prior work on local graph clustering mostly falls into two categories with numerical and comb...More

Code:

Data:

0
Introduction
  • Graphs are ubiquitous when it comes to modeling relationships among entities, e.g. social [30] and biology [32] networks, and graphs arising from modern applications are massive yet rich in small-scale local structures [20, 16, 9].
  • The authors' first main result is a novel theoretical analysis for local graph clustering using the optimal solution of p-norm diffusion.
  • The authors show that a randomized coordinate descent method can obtain an accurate solution of p-norm diffusion for p ≥ 2 in strongly local running time.
Highlights
  • Graphs are ubiquitous when it comes to modeling relationships among entities, e.g. social [30] and biology [32] networks, and graphs arising from modern applications are massive yet rich in small-scale local structures [20, 16, 9]
  • Exploiting such local structures is of central importance in many areas of machine learning and applied mathematics, e.g. community detection in networks [25, 35, 20, 16] and PageRank-based spectral ranking in web ranking [27, 12]
  • Somewhat more formally, we consider local graph clustering as the task of finding a community-like cluster around a given set of seed nodes, where nodes in the cluster are densely connected to each other while relatively isolated to the rest of the graph
  • Local algorithms for local graph clustering are predominantly based on the idea of diffusion, which is the generic process of spreading mass among vertices by sending mass along edges
  • The most popular diffusion methods are spectral methods based on the connection between graph structures and the algebraic properties of the spectrum of matrices associated with the graph [22, 23, 6, 31]
Results
  • Given a graph G with signed incidence matrix B, and two functions ∆, T : V → R≥0, the authors propose the following pair of convex optimization problems, which are the p-norm flow diffusion minimize f p s.t. BT f + ∆ ≤ T
  • The authors discuss the optimal solutions of the diffusion problem in the context of local graph clustering.
  • To specify a particular diffusion problem and its dual, the authors need to provide the source mass ∆, and recall the authors always set the sink capacity T (v) = deg(v).
  • The authors will discuss the choice of δ shortly, but the authors start with a simple lemma on the locality of the optimal solutions for the primal and dual problems.
  • When C is a cut of low conductance, any feasible routing must incur a large cost since vol(C) amount of mass has to get out of C using a relatively small number of discharging edges.
  • Any edge e = (u, v) crossing a level cut Sh must have dual values x∗(u), x∗(v) on different sides of h, having non-zero length l(e), which meansl(e) is at least 1/vol(C)1/q.
  • In the context of p-norm flow diffusion, coordinate method enjoys a natural combinatorial interpretation3: each coordinate update corresponds to routing mass through incident edges and distributing mass across its neighbours.
  • Note that fμ(x) is not strongly convex in general, but locality gives them strong convexity: when restricting fμ to the iterates generated by Algorithm 1, fμ has a strong convexity parameter γ which is the minimum eigenvalue of the sub-matrix of the Laplacian defined by the nodes in supp(x∗μ) [8], multiplied by a positive weight constant, and satisfies
Conclusion
  • Lipschitz continuity, strong convexity, and error bound (9) give them linear convergence rate to the dual problem (8) and the following running time guarantee.
  • In all subsequent experiments when the authors compare recovery results with 1regularized PageRank and nonlinear power diffusion, the authors always start the diffusion process from one seed node, as this is the most common practice for semi-supervised local clustering tasks.
  • P-norm flow diffusion methods have both the best F1 measure and conductance result.
Summary
  • Graphs are ubiquitous when it comes to modeling relationships among entities, e.g. social [30] and biology [32] networks, and graphs arising from modern applications are massive yet rich in small-scale local structures [20, 16, 9].
  • The authors' first main result is a novel theoretical analysis for local graph clustering using the optimal solution of p-norm diffusion.
  • The authors show that a randomized coordinate descent method can obtain an accurate solution of p-norm diffusion for p ≥ 2 in strongly local running time.
  • Given a graph G with signed incidence matrix B, and two functions ∆, T : V → R≥0, the authors propose the following pair of convex optimization problems, which are the p-norm flow diffusion minimize f p s.t. BT f + ∆ ≤ T
  • The authors discuss the optimal solutions of the diffusion problem in the context of local graph clustering.
  • To specify a particular diffusion problem and its dual, the authors need to provide the source mass ∆, and recall the authors always set the sink capacity T (v) = deg(v).
  • The authors will discuss the choice of δ shortly, but the authors start with a simple lemma on the locality of the optimal solutions for the primal and dual problems.
  • When C is a cut of low conductance, any feasible routing must incur a large cost since vol(C) amount of mass has to get out of C using a relatively small number of discharging edges.
  • Any edge e = (u, v) crossing a level cut Sh must have dual values x∗(u), x∗(v) on different sides of h, having non-zero length l(e), which meansl(e) is at least 1/vol(C)1/q.
  • In the context of p-norm flow diffusion, coordinate method enjoys a natural combinatorial interpretation3: each coordinate update corresponds to routing mass through incident edges and distributing mass across its neighbours.
  • Note that fμ(x) is not strongly convex in general, but locality gives them strong convexity: when restricting fμ to the iterates generated by Algorithm 1, fμ has a strong convexity parameter γ which is the minimum eigenvalue of the sub-matrix of the Laplacian defined by the nodes in supp(x∗μ) [8], multiplied by a positive weight constant, and satisfies
  • Lipschitz continuity, strong convexity, and error bound (9) give them linear convergence rate to the dual problem (8) and the following running time guarantee.
  • In all subsequent experiments when the authors compare recovery results with 1regularized PageRank and nonlinear power diffusion, the authors always start the diffusion process from one seed node, as this is the most common practice for semi-supervised local clustering tasks.
  • P-norm flow diffusion methods have both the best F1 measure and conductance result.
Tables
  • Table1: Results for real-world graphs dataset
  • Table2: Summary of real-world graphs dataset number of nodes number of edges description
  • Table3: Filtered “ground truth” clusters for real-world graphs dataset
Download tables as Excel
Reference
  • R. Andersen, F. Chung, and K. Lang. Local graph partitioning using pagerank vectors. FOCS ’06 Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, pages 475–486, 2006.
    Google ScholarLocate open access versionFindings
  • R. Andersen, S. O. Gharan, Y. Peres, and L. Trevisan. Almost optimal local graph clustering using evolving sets. Journal of the ACM, 63(2), 2016.
    Google ScholarLocate open access versionFindings
  • R. Andersen and K. J. Lang. An algorithm for improving graph partitions. Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms, pages 651–660, 2008.
    Google ScholarLocate open access versionFindings
  • Reid Andersen and Yuval Peres. Finding sparse cuts locally using evolving sets. In STOC 2009, pages 235–244, 2009.
    Google ScholarLocate open access versionFindings
  • Shoshana D Brown, John A Gerlt, Jennifer L Seffernick, and Patricia C Babbitt. A gold standard set of mechanistically diverse enzyme superfamilies. Genome biology, 7(1):R8, 2006.
    Google ScholarLocate open access versionFindings
  • F. Chung. The heat kernel as the PageRank of a graph. Proceedings of the National Academy of Sciences, 104(50):19735–19740, 2007.
    Google ScholarLocate open access versionFindings
  • F. R. K. Chung. A local graph partitioning algorithm using heat kernel pagerank. Internet Mathematics, 6(3):315–330, 2009.
    Google ScholarLocate open access versionFindings
  • Fan Chung. Random walks and local cuts in graphs. Linear Algebra and its Applications, 423(1):22 – 32, 2007. Special Issue devoted to papers presented at the Aveiro Workshop on Graph Spectra.
    Google ScholarLocate open access versionFindings
  • S. Fortunato and D. Hric. Community detection in networks: A user guide. Physics Reports, 659(11):1–44, 2016.
    Google ScholarLocate open access versionFindings
  • K. Fountoulakis, D. F. Gleich, and M. W. Mahoney. An optimization approach to locallybiased graph algorithms. Proceedings of the IEEE, 105(2):256–272, 2017.
    Google ScholarLocate open access versionFindings
  • K. Fountoulakis, F. Roosta-Khorasani, J. Shun, X. Cheng, and M. W. Mahoney. Variational perspective on local graph clustering. Mathematical Programming, 174:pages553–573, 2017.
    Google ScholarLocate open access versionFindings
  • D. F. Gleich. PageRank beyond the web. SIAM Review, 57(3):321–363, 2015.
    Google ScholarLocate open access versionFindings
  • Michael Grant and Stephen Boyd. CVX: Matlab software for disciplined convex programming, version 2.1. http://cvxr.com/cvx, March 2014.
    Findings
  • S. Guattery and G.L. Miller. On the quality of spectral separators. SIAM Journal on Matrix Analysis and Applications, 19:701–719, 1998.
    Google ScholarLocate open access versionFindings
  • R. Ibrahim and D. Gleich. Nonlinear diffusion for community detection and semi-supervised learning. WWW’19: The World Wide Web Conference, pages 739–750, 2019.
    Google ScholarLocate open access versionFindings
  • L. G. S. Jeub, P. Balachandran, M. A. Porter, P. J. Mucha, and M. W. Mahoney. Think locally, act locally: Detection of small, medium-sized, and large communities in large networks. Physical Review E, 91:012821, 2015.
    Google ScholarLocate open access versionFindings
  • K. Kloster and D. F. Gleich. Heat kernel based community detection. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1386–1395, 2014.
    Google ScholarLocate open access versionFindings
  • I. M. Kloumann and J. M. Kleinberg. Community membership identification from small seed sets. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1366–1375, 2014. 13
    Google ScholarLocate open access versionFindings
  • Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. Benchmark graphs for testing community detection algorithms. Physical review. E, Statistical, nonlinear, and soft matter physics, 78:046110, 11 2008.
    Google ScholarLocate open access versionFindings
  • J. Leskovec, K.J. Lang, A. Dasgupta, and M.W. Mahoney. Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters. Internet Mathematics, 6(1):29–123, 2009.
    Google ScholarLocate open access versionFindings
  • Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data, June 2014.
    Findings
  • L. Lovasz and M. Simonovits. The mixing rate of Markov chains, an isoperimetric inequality, and computing the volume. Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science, pages 346–354, 1990.
    Google ScholarLocate open access versionFindings
  • Laszlo Lovasz and Miklos Simonovits. Random walks in a convex body and an improved volume algorithm. Random Struct. Algorithms, 4(4):359–412, 1993.
    Google ScholarLocate open access versionFindings
  • M. W. Mahoney, L. Orecchia, and N. K. Vishnoi. A local spectral method for graphs: with applications to improving graph partitions and exploring data graphs locally. Journal of Machine Learning Research, 13:2339–2365, 2012.
    Google ScholarLocate open access versionFindings
  • A.Y. Ng, M.I. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In NIPS ’01: Proceedings of the 15th Annual Conference on Advances in Neural Information Processing Systems, 2001.
    Google ScholarLocate open access versionFindings
  • L. Orecchia and Z. A. Zhu. Flow-based algorithms for local graph clustering. In Proceedings of the 25th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1267–1286, 2014.
    Google ScholarLocate open access versionFindings
  • Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford InfoLab, 1999.
    Google ScholarFindings
  • P. Shi, K. He, D. Bindel, and J. Hopcroft. Local Lanczos spectral approximation for community detection. In Proceedings of ECML-PKDD, September 2017.
    Google ScholarLocate open access versionFindings
  • D. A. Spielman and S. H. Teng. A local clustering algorithm for massive graphs and its application to nearly linear time graph partitioning. SIAM Journal on Scientific Computing, 42(1):1–26, 2013.
    Google ScholarLocate open access versionFindings
  • A. L. Traud, P. J. Mucha, and M. A. Porter. Social structure of facebook networks. Physica A: Statistical Mechanics and its Applications, 391(16):4165–4180, 2012.
    Google ScholarLocate open access versionFindings
  • A. Tsiatas. Diffusion and clustering on large graphs. PhD Thesis, University of California, San Diego, 2012.
    Google ScholarFindings
  • N. Tuncbag, P. Milani, J. L. Pokorny, H. Johnson, T. T. Sio, S. Dalin, D. O. Iyekegbe, F. M. White, J. N. Sarkaria, and E. Fraenkel. Network modeling identifies patient-specific pathways in glioblastoma. Scientific Reports, 6:28668, 2016.
    Google ScholarLocate open access versionFindings
  • U. von Luxburg. A tutorial on spectral clustering. Technical Report 149, Max Plank Institute for Biological Cybernetics, August 2006.
    Google ScholarFindings
  • D. Wang, K. Fountoulakis, M. Henzinger, M. W. Mahoney, and S. Rao. Capacity releasing diffusion for speed and locality. Proceedings of the 34th International Conference on Machine Learning, 70:3607–2017, 2017.
    Google ScholarLocate open access versionFindings
  • S. White and P. Smyth. A spectral clustering approach to finding communities in graphs. In SDM ’05: Proceedings of the 5th SIAM International Conference on Data Mining, pages 76–84, 2005. 14
    Google ScholarLocate open access versionFindings
  • Jaewon Yang and Jure Leskovec. Defining and evaluating network communities based on ground-truth. CoRR, abs/1205.6233, 2012.
    Findings
  • Zeyuan Allen Zhu, Silvio Lattanzi, and Vahab S. Mirrokni. A local algorithm for finding well-connected clusters. In ICML 2013, pages 396–404, 2013.
    Google ScholarLocate open access versionFindings
  • 0. Let x, λ, ybe a solution to the latter optimality conditions.
    Google ScholarFindings
  • 1. Initially, all nodes have height 0, i.e., x(v) = 0 for all v. 2. While ex(v; x, μ) > 0 for some node v: (a) Pick any v where ex(v; x, μ) > 0 uniformly at random. (b) Route the flow around node v by raising it to a new height x(v) = x(v) + μq−2 ex(v; x, μ). deg(v)
    Google ScholarFindings
Full Text
Your rating :
0

 

Tags
Comments