# p-Norm Flow Diffusion for Local Graph Clustering

international conference on machine learning, 2020.

Keywords:

Weibo:

Abstract:

Local graph clustering and the closely related seed set expansion problem are primitives on graphs that are central to a wide range of analytic and learning tasks such as local clustering, community detection, nodes ranking and feature inference. Prior work on local graph clustering mostly falls into two categories with numerical and comb...More

Code:

Data:

Introduction

- Graphs are ubiquitous when it comes to modeling relationships among entities, e.g. social [30] and biology [32] networks, and graphs arising from modern applications are massive yet rich in small-scale local structures [20, 16, 9].
- The authors' first main result is a novel theoretical analysis for local graph clustering using the optimal solution of p-norm diffusion.
- The authors show that a randomized coordinate descent method can obtain an accurate solution of p-norm diffusion for p ≥ 2 in strongly local running time.

Highlights

- Graphs are ubiquitous when it comes to modeling relationships among entities, e.g. social [30] and biology [32] networks, and graphs arising from modern applications are massive yet rich in small-scale local structures [20, 16, 9]
- Exploiting such local structures is of central importance in many areas of machine learning and applied mathematics, e.g. community detection in networks [25, 35, 20, 16] and PageRank-based spectral ranking in web ranking [27, 12]
- Somewhat more formally, we consider local graph clustering as the task of finding a community-like cluster around a given set of seed nodes, where nodes in the cluster are densely connected to each other while relatively isolated to the rest of the graph
- Local algorithms for local graph clustering are predominantly based on the idea of diffusion, which is the generic process of spreading mass among vertices by sending mass along edges
- The most popular diffusion methods are spectral methods based on the connection between graph structures and the algebraic properties of the spectrum of matrices associated with the graph [22, 23, 6, 31]

Results

- Given a graph G with signed incidence matrix B, and two functions ∆, T : V → R≥0, the authors propose the following pair of convex optimization problems, which are the p-norm flow diffusion minimize f p s.t. BT f + ∆ ≤ T
- The authors discuss the optimal solutions of the diffusion problem in the context of local graph clustering.
- To specify a particular diffusion problem and its dual, the authors need to provide the source mass ∆, and recall the authors always set the sink capacity T (v) = deg(v).
- The authors will discuss the choice of δ shortly, but the authors start with a simple lemma on the locality of the optimal solutions for the primal and dual problems.
- When C is a cut of low conductance, any feasible routing must incur a large cost since vol(C) amount of mass has to get out of C using a relatively small number of discharging edges.
- Any edge e = (u, v) crossing a level cut Sh must have dual values x∗(u), x∗(v) on different sides of h, having non-zero length l(e), which meansl(e) is at least 1/vol(C)1/q.
- In the context of p-norm flow diffusion, coordinate method enjoys a natural combinatorial interpretation3: each coordinate update corresponds to routing mass through incident edges and distributing mass across its neighbours.
- Note that fμ(x) is not strongly convex in general, but locality gives them strong convexity: when restricting fμ to the iterates generated by Algorithm 1, fμ has a strong convexity parameter γ which is the minimum eigenvalue of the sub-matrix of the Laplacian defined by the nodes in supp(x∗μ) [8], multiplied by a positive weight constant, and satisfies

Conclusion

- Lipschitz continuity, strong convexity, and error bound (9) give them linear convergence rate to the dual problem (8) and the following running time guarantee.
- In all subsequent experiments when the authors compare recovery results with 1regularized PageRank and nonlinear power diffusion, the authors always start the diffusion process from one seed node, as this is the most common practice for semi-supervised local clustering tasks.
- P-norm flow diffusion methods have both the best F1 measure and conductance result.

Summary

- Graphs are ubiquitous when it comes to modeling relationships among entities, e.g. social [30] and biology [32] networks, and graphs arising from modern applications are massive yet rich in small-scale local structures [20, 16, 9].
- The authors' first main result is a novel theoretical analysis for local graph clustering using the optimal solution of p-norm diffusion.
- The authors show that a randomized coordinate descent method can obtain an accurate solution of p-norm diffusion for p ≥ 2 in strongly local running time.
- Given a graph G with signed incidence matrix B, and two functions ∆, T : V → R≥0, the authors propose the following pair of convex optimization problems, which are the p-norm flow diffusion minimize f p s.t. BT f + ∆ ≤ T
- The authors discuss the optimal solutions of the diffusion problem in the context of local graph clustering.
- To specify a particular diffusion problem and its dual, the authors need to provide the source mass ∆, and recall the authors always set the sink capacity T (v) = deg(v).
- The authors will discuss the choice of δ shortly, but the authors start with a simple lemma on the locality of the optimal solutions for the primal and dual problems.
- When C is a cut of low conductance, any feasible routing must incur a large cost since vol(C) amount of mass has to get out of C using a relatively small number of discharging edges.
- Any edge e = (u, v) crossing a level cut Sh must have dual values x∗(u), x∗(v) on different sides of h, having non-zero length l(e), which meansl(e) is at least 1/vol(C)1/q.
- In the context of p-norm flow diffusion, coordinate method enjoys a natural combinatorial interpretation3: each coordinate update corresponds to routing mass through incident edges and distributing mass across its neighbours.
- Note that fμ(x) is not strongly convex in general, but locality gives them strong convexity: when restricting fμ to the iterates generated by Algorithm 1, fμ has a strong convexity parameter γ which is the minimum eigenvalue of the sub-matrix of the Laplacian defined by the nodes in supp(x∗μ) [8], multiplied by a positive weight constant, and satisfies
- Lipschitz continuity, strong convexity, and error bound (9) give them linear convergence rate to the dual problem (8) and the following running time guarantee.
- In all subsequent experiments when the authors compare recovery results with 1regularized PageRank and nonlinear power diffusion, the authors always start the diffusion process from one seed node, as this is the most common practice for semi-supervised local clustering tasks.
- P-norm flow diffusion methods have both the best F1 measure and conductance result.

- Table1: Results for real-world graphs dataset
- Table2: Summary of real-world graphs dataset number of nodes number of edges description
- Table3: Filtered “ground truth” clusters for real-world graphs dataset

Reference

- R. Andersen, F. Chung, and K. Lang. Local graph partitioning using pagerank vectors. FOCS ’06 Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, pages 475–486, 2006.
- R. Andersen, S. O. Gharan, Y. Peres, and L. Trevisan. Almost optimal local graph clustering using evolving sets. Journal of the ACM, 63(2), 2016.
- R. Andersen and K. J. Lang. An algorithm for improving graph partitions. Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms, pages 651–660, 2008.
- Reid Andersen and Yuval Peres. Finding sparse cuts locally using evolving sets. In STOC 2009, pages 235–244, 2009.
- Shoshana D Brown, John A Gerlt, Jennifer L Seffernick, and Patricia C Babbitt. A gold standard set of mechanistically diverse enzyme superfamilies. Genome biology, 7(1):R8, 2006.
- F. Chung. The heat kernel as the PageRank of a graph. Proceedings of the National Academy of Sciences, 104(50):19735–19740, 2007.
- F. R. K. Chung. A local graph partitioning algorithm using heat kernel pagerank. Internet Mathematics, 6(3):315–330, 2009.
- Fan Chung. Random walks and local cuts in graphs. Linear Algebra and its Applications, 423(1):22 – 32, 2007. Special Issue devoted to papers presented at the Aveiro Workshop on Graph Spectra.
- S. Fortunato and D. Hric. Community detection in networks: A user guide. Physics Reports, 659(11):1–44, 2016.
- K. Fountoulakis, D. F. Gleich, and M. W. Mahoney. An optimization approach to locallybiased graph algorithms. Proceedings of the IEEE, 105(2):256–272, 2017.
- K. Fountoulakis, F. Roosta-Khorasani, J. Shun, X. Cheng, and M. W. Mahoney. Variational perspective on local graph clustering. Mathematical Programming, 174:pages553–573, 2017.
- D. F. Gleich. PageRank beyond the web. SIAM Review, 57(3):321–363, 2015.
- Michael Grant and Stephen Boyd. CVX: Matlab software for disciplined convex programming, version 2.1. http://cvxr.com/cvx, March 2014.
- S. Guattery and G.L. Miller. On the quality of spectral separators. SIAM Journal on Matrix Analysis and Applications, 19:701–719, 1998.
- R. Ibrahim and D. Gleich. Nonlinear diffusion for community detection and semi-supervised learning. WWW’19: The World Wide Web Conference, pages 739–750, 2019.
- L. G. S. Jeub, P. Balachandran, M. A. Porter, P. J. Mucha, and M. W. Mahoney. Think locally, act locally: Detection of small, medium-sized, and large communities in large networks. Physical Review E, 91:012821, 2015.
- K. Kloster and D. F. Gleich. Heat kernel based community detection. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1386–1395, 2014.
- I. M. Kloumann and J. M. Kleinberg. Community membership identification from small seed sets. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1366–1375, 2014. 13
- Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. Benchmark graphs for testing community detection algorithms. Physical review. E, Statistical, nonlinear, and soft matter physics, 78:046110, 11 2008.
- J. Leskovec, K.J. Lang, A. Dasgupta, and M.W. Mahoney. Community structure in large networks: Natural cluster sizes and the absence of large well-defined clusters. Internet Mathematics, 6(1):29–123, 2009.
- Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data, June 2014.
- L. Lovasz and M. Simonovits. The mixing rate of Markov chains, an isoperimetric inequality, and computing the volume. Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science, pages 346–354, 1990.
- Laszlo Lovasz and Miklos Simonovits. Random walks in a convex body and an improved volume algorithm. Random Struct. Algorithms, 4(4):359–412, 1993.
- M. W. Mahoney, L. Orecchia, and N. K. Vishnoi. A local spectral method for graphs: with applications to improving graph partitions and exploring data graphs locally. Journal of Machine Learning Research, 13:2339–2365, 2012.
- A.Y. Ng, M.I. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In NIPS ’01: Proceedings of the 15th Annual Conference on Advances in Neural Information Processing Systems, 2001.
- L. Orecchia and Z. A. Zhu. Flow-based algorithms for local graph clustering. In Proceedings of the 25th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1267–1286, 2014.
- Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford InfoLab, 1999.
- P. Shi, K. He, D. Bindel, and J. Hopcroft. Local Lanczos spectral approximation for community detection. In Proceedings of ECML-PKDD, September 2017.
- D. A. Spielman and S. H. Teng. A local clustering algorithm for massive graphs and its application to nearly linear time graph partitioning. SIAM Journal on Scientific Computing, 42(1):1–26, 2013.
- A. L. Traud, P. J. Mucha, and M. A. Porter. Social structure of facebook networks. Physica A: Statistical Mechanics and its Applications, 391(16):4165–4180, 2012.
- A. Tsiatas. Diffusion and clustering on large graphs. PhD Thesis, University of California, San Diego, 2012.
- N. Tuncbag, P. Milani, J. L. Pokorny, H. Johnson, T. T. Sio, S. Dalin, D. O. Iyekegbe, F. M. White, J. N. Sarkaria, and E. Fraenkel. Network modeling identifies patient-specific pathways in glioblastoma. Scientific Reports, 6:28668, 2016.
- U. von Luxburg. A tutorial on spectral clustering. Technical Report 149, Max Plank Institute for Biological Cybernetics, August 2006.
- D. Wang, K. Fountoulakis, M. Henzinger, M. W. Mahoney, and S. Rao. Capacity releasing diffusion for speed and locality. Proceedings of the 34th International Conference on Machine Learning, 70:3607–2017, 2017.
- S. White and P. Smyth. A spectral clustering approach to finding communities in graphs. In SDM ’05: Proceedings of the 5th SIAM International Conference on Data Mining, pages 76–84, 2005. 14
- Jaewon Yang and Jure Leskovec. Defining and evaluating network communities based on ground-truth. CoRR, abs/1205.6233, 2012.
- Zeyuan Allen Zhu, Silvio Lattanzi, and Vahab S. Mirrokni. A local algorithm for finding well-connected clusters. In ICML 2013, pages 396–404, 2013.
- 0. Let x, λ, ybe a solution to the latter optimality conditions.
- 1. Initially, all nodes have height 0, i.e., x(v) = 0 for all v. 2. While ex(v; x, μ) > 0 for some node v: (a) Pick any v where ex(v; x, μ) > 0 uniformly at random. (b) Route the flow around node v by raising it to a new height x(v) = x(v) + μq−2 ex(v; x, μ). deg(v)

Full Text

Tags

Comments