AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
This paper presents GBP, a scalable Graph Neural Networks based on localized bidirectional propagation

Scalable Graph Neural Networks via Bidirectional Propagation

NIPS 2020, (2020)

被引用0|浏览215
EI
下载 PDF 全文
引用
微博一下

摘要

Graph Neural Networks (GNN) is an emerging field for learning on non-Euclidean data. Recently, there has been increased interest in designing GNN that scales to large graphs. Most existing methods use "graph sampling" or "layer-wise sampling" techniques to reduce training time. However, these methods still suffer from degrading performa...更多

代码

数据

简介
  • The field of Graph Neural Networks (GNNs) has drawn increasing attention due to its wide range of applications such as social analysis [23, 20, 28], biology [10, 26], recommendation systems [36], and computer vision [39, 7, 13].
  • On the other hand, training GCN with mini-batches is difficult, as the neighborhood size could grow exponentially with the number of layers.
  • These techniques can be broadly divided into three categories: 1) Layer-wise sampling methods: GraphSAGE [11] proposes a neighbor-sampling method to sample a fixed number of neighbors for each node.
  • FastGCN [5] samples nodes of each layer independently
重点内容
  • The field of Graph Neural Networks (GNNs) has drawn increasing attention due to its wide range of applications such as social analysis [23, 20, 28], biology [10, 26], recommendation systems [36], and computer vision [39, 7, 13]
  • The vanilla Graph Convolutional Network (GCN) uses a full-batch training process and stores each node’s representation in the GPU memory, which leads to limited scalability
  • VRGCN [6] leverages historical activations to restrict the number of sampled nodes and reduce the variance of sampling
  • To reduce the time complexity, we propose approximating the Generalized PageRank matrix P with a localized bidirectional propagation algorithm from both the training/testing nodes and the feature vectors
  • This paper presents GBP, a scalable GNN based on localized bidirectional propagation
  • GBP is the first method that can scale to billion-edge networks on a single machine
方法
  • GCN GAT APPNP GDC SGC LADIES PPRGo GraphSAINT

    URL https://github.com/rusty1s/pytorch_geometric https://github.com/rusty1s/pytorch_geometric https://github.com/rusty1s/pytorch_geometric https://github.com/klicperajo/gdc https://github.com/Tiiiger/SGC https://github.com/acbull/LADIES https://github.com/TUM-DAML/pprgo_pytorch https://github.com/GraphSAINT/GraphSAINT Commit

    5692a8 5692a8 5692a8 14333f 6c450f c7f987 d9f991 cd31c3.
结果
  • The authors observe that GBP can achieve a significantly higher F1-score with less running time.
结论
  • This paper presents GBP, a scalable GNN based on localized bidirectional propagation.
  • The bidirectional propagation process computes a Generalized PageRank matrix that can express various existing graph convolutions.
  • Extensive experiments on real-world graphs show that GBP obtains significant improvement over the state-of-the-art methods in terms of efficiency and performance.
  • GBP is the first method that can scale to billion-edge networks on a single machine.
  • An interesting direction is to extend GBP to heterogeneous networks
总结
  • Introduction:

    The field of Graph Neural Networks (GNNs) has drawn increasing attention due to its wide range of applications such as social analysis [23, 20, 28], biology [10, 26], recommendation systems [36], and computer vision [39, 7, 13].
  • On the other hand, training GCN with mini-batches is difficult, as the neighborhood size could grow exponentially with the number of layers.
  • These techniques can be broadly divided into three categories: 1) Layer-wise sampling methods: GraphSAGE [11] proposes a neighbor-sampling method to sample a fixed number of neighbors for each node.
  • FastGCN [5] samples nodes of each layer independently
  • Methods:

    GCN GAT APPNP GDC SGC LADIES PPRGo GraphSAINT

    URL https://github.com/rusty1s/pytorch_geometric https://github.com/rusty1s/pytorch_geometric https://github.com/rusty1s/pytorch_geometric https://github.com/klicperajo/gdc https://github.com/Tiiiger/SGC https://github.com/acbull/LADIES https://github.com/TUM-DAML/pprgo_pytorch https://github.com/GraphSAINT/GraphSAINT Commit

    5692a8 5692a8 5692a8 14333f 6c450f c7f987 d9f991 cd31c3.
  • Results:

    The authors observe that GBP can achieve a significantly higher F1-score with less running time.
  • Conclusion:

    This paper presents GBP, a scalable GNN based on localized bidirectional propagation.
  • The bidirectional propagation process computes a Generalized PageRank matrix that can express various existing graph convolutions.
  • Extensive experiments on real-world graphs show that GBP obtains significant improvement over the state-of-the-art methods in terms of efficiency and performance.
  • GBP is the first method that can scale to billion-edge networks on a single machine.
  • An interesting direction is to extend GBP to heterogeneous networks
表格
  • Table1: Summary of time complexity for GNN training and inference
  • Table2: Dataset statistics
  • Table3: Hyper-parameters of GBP. rmax is the Reverse Push Threshold, w is the number of random walks from the training nodes, w is the weight sequence, r is the Laplacian parameter in the convolutional matrix Dr−1AD−r
  • Table4: Results on Cora, Citeseer and Pubmed
  • Table5: Results of inductive learning with scalable GNNs
  • Table6: Results for semi-supervised learning on Friendster
  • Table7: URLs of baseline codes
Download tables as Excel
基金
  • Acknowledgments and Disclosure of Funding Ji-Rong Wen was supported by National Natural Science Foundation of China (NSFC) No.61832017, and by Beijing Outstanding Young Scientist Program NO
  • Zhewei Wei was supported by NSFC No 61972401 and No 61932001, by the Fundamental Research Funds for the Central Universities and the Research Funds of Renmin University of China under Grant 18XNLG21, and by Alibaba Group through Alibaba Innovative Research Program
  • Ye Yuan was supported by NSFC No 61932004 and No 61622202, and by FRFCU No N181605012
  • Xiaoyong Du was supported by NSFC No U1711261
研究对象与分析
open graph datasets with different size: 7
Datasets. We use seven open graph datasets with different size: three citation networks Cora, Citeser and Pubmed [25], a Protein-Protein interaction network PPI [11], a customer interaction. Cora Citeseer Pubmed PPI Yelp Amazon Friendster

across all datasets: 4
On the Friendster dataset, where the features are random noises, we use both Personalized PageRank and transition probability (wL = 1, w0 =, . . . , = wL−1 = 0) for GBP. We set L = 4 across all datasets. Table 3 summaries other hyper-parameters of GBP on different datasets

large datasets: 3
Inductive learning on medium to large graphs. Table 5 reports the F1-score and running time (precomputation + training) of each method with various depths on three large datasets PPI, Yelp, and Amazon. For each dataset, we set the hidden dimension to be the same across all methods: 2048(PPI), 2048(Yelp), and 1024(Amazon)

引用论文
  • ABU-EL-HAIJA, S., PEROZZI, B., KAPOOR, A., ALIPOURFARD, N., LERMAN, K., HARUTYUNYAN, H., STEEG, G. V., AND GALSTYAN, A. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In ICML (2019).
    Google ScholarLocate open access versionFindings
  • ANDERSEN, R., CHUNG, F. R. K., AND LANG, K. J. Local graph partitioning using pagerank vectors. In FOCS (2006), IEEE Computer Society, pp. 475–486.
    Google ScholarLocate open access versionFindings
  • BANERJEE, S., AND LOFGREN, P. Fast bidirectional probability estimation in markov models. In NIPS (2015), pp. 1423–1431.
    Google ScholarLocate open access versionFindings
  • BOJCHEVSKI, A., KLICPERA, J., PEROZZI, B., KAPOOR, A., BLAIS, M., RÓZEMBERCZKI, B., LUKASIK, M., AND GÜNNEMANN, S. Scaling graph neural networks with approximate pagerank. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, NY, USA, 2020), ACM.
    Google ScholarLocate open access versionFindings
  • CHEN, J., MA, T., AND XIAO, C. Fastgcn: Fast learning with graph convolutional networks via importance sampling. In ICLR (2018).
    Google ScholarLocate open access versionFindings
  • CHEN, J., ZHU, J., AND SONG, L. Stochastic training of graph convolutional networks with variance reduction. In ICML (2018).
    Google ScholarLocate open access versionFindings
  • CHEN, Z., WEI, X., WANG, P., AND GUO, Y. Multi-label image recognition with graph convolutional networks. In CVPR (2019).
    Google ScholarLocate open access versionFindings
  • CHIANG, W., LIU, X., SI, S., LI, Y., BENGIO, S., AND HSIEH, C. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In KDD (2019), ACM, pp. 257–266.
    Google ScholarLocate open access versionFindings
  • CHUNG, F. R. K., AND LU, L. Survey: Concentration inequalities and martingale inequalities: A survey. Internet Math. 3, 1 (2006), 79–127.
    Google ScholarLocate open access versionFindings
  • FOUT, A., BYRD, J., SHARIAT, B., AND BEN-HUR, A. Protein interface prediction using graph convolutional networks. In NeurIPS (2017), pp. 6530–6539.
    Google ScholarLocate open access versionFindings
  • HAMILTON, W. L., YING, R., AND LESKOVEC, J. Inductive representation learning on large graphs. In NeurIPS (2017).
    Google ScholarLocate open access versionFindings
  • HE, K., ZHANG, X., REN, S., AND SUN, J. Deep residual learning for image recognition. In CVPR (2016), pp. 770–778.
    Google ScholarLocate open access versionFindings
  • HOSSEINI, B., MONTAGNE, R., AND HAMMER, B. Deep-aligned convolutional neural network for skeleton-based action recognition and segmentation. In Data Science and Engineering (2020).
    Google ScholarLocate open access versionFindings
  • KARYPIS, G., AND KUMAR, V. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Scientific Computing 20, 1 (1998), 359–392.
    Google ScholarLocate open access versionFindings
  • KIPF, T. N., AND WELLING, M. Semi-supervised classification with graph convolutional networks. In ICLR (2017).
    Google ScholarLocate open access versionFindings
  • KLICPERA, J., BOJCHEVSKI, A., AND GÜNNEMANN, S. Predict then propagate: Graph neural networks meet personalized pagerank. In ICLR (2019).
    Google ScholarLocate open access versionFindings
  • KLICPERA, J., WEISSENBERGER, S., AND GÜNNEMANN, S. Diffusion improves graph learning. In NeurIPS (2019), pp. 13333–13345.
    Google ScholarLocate open access versionFindings
  • KLOSTER, K., AND GLEICH, D. F. Heat kernel based community detection. In KDD (2014), ACM, pp. 1386–1395.
    Google ScholarLocate open access versionFindings
  • LESKOVEC, J., LANG, K. J., AND MAHONEY, M. W. Empirical comparison of algorithms for network community detection. In WWW (2010), ACM, pp. 631–640.
    Google ScholarLocate open access versionFindings
  • LI, C., AND GOLDWASSER, D. Encoding social information with graph convolutional networks forpolitical perspective detection in news media. In ACL (2019).
    Google ScholarLocate open access versionFindings
  • LI, P., CHIEN, I. E., AND MILENKOVIC, O. Optimizing generalized pagerank methods for seed-expansion community detection. In NeurIPS (2019), H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. B. Fox, and R. Garnett, Eds., pp. 11705–11716.
    Google ScholarFindings
  • LU, H., HUANG, S. H., YE, T., AND GUO, X. Graph star net for generalized multi-task learning. CoRR abs/1906.12330 (2019).
    Findings
  • QIU, J., TANG, J., MA, H., DONG, Y., WANG, K., AND TANG, J. Deepinf: Social influence prediction with deep learning. In KDD (2018), ACM, pp. 2110–2119.
    Google ScholarLocate open access versionFindings
  • RONG, Y., HUANG, W., XU, T., AND HUANG, J. Dropedge: Towards deep graph convolutional networks on node classification. In ICLR (2020).
    Google ScholarLocate open access versionFindings
  • SEN, P., NAMATA, G., BILGIC, M., GETOOR, L., GALLAGHER, B., AND ELIASSI-RAD, T. Collective classification in network data. AI Magazine 29, 3 (2008), 93–106.
    Google ScholarLocate open access versionFindings
  • SHANG, J., XIAO, C., MA, T., LI, H., AND SUN, J. Gamenet: Graph augmented memory networks for recommending medication combination. In AAAI (2019).
    Google ScholarLocate open access versionFindings
  • THEKUMPARAMPIL, K. K., WANG, C., OH, S., AND LI, L. Attention-based graph neural network for semi-supervised learning. CoRR abs/1803.03735 (2018).
    Findings
  • TONG, P., ZHANG, Q., AND YAO, J. Leveraging domain context for question answering over knowledge graph. In Data Science and Engineering (2019).
    Google ScholarLocate open access versionFindings
  • VELIC KOVIC, P., CUCURULL, G., CASANOVA, A., ROMERO, A., LIÒ, P., AND BENGIO, Y. Graph Attention Networks. ICLR (2018).
    Google ScholarLocate open access versionFindings
  • WU, F., SOUZA, A., ZHANG, T., FIFTY, C., YU, T., AND WEINBERGER, K. Simplifying graph convolutional networks. In ICML (2019), pp. 6861–6871.
    Google ScholarLocate open access versionFindings
  • XU, K., HU, W., LESKOVEC, J., AND JEGELKA, S. How powerful are graph neural networks? In ICLR (2019).
    Google ScholarLocate open access versionFindings
  • XU, K., LI, C., TIAN, Y., SONOBE, T., KAWARABAYASHI, K., AND JEGELKA, S. Representation learning on graphs with jumping knowledge networks. In ICML (2018).
    Google ScholarLocate open access versionFindings
  • YANG, C., XIAO, Y., ZHANG, Y., SUN, Y., AND HAN, J. Heterogeneous network representation learning: Survey, benchmark, evaluation, and beyond. arXiv preprint arXiv:2004.00216 (2020).
    Findings
  • YANG, J., AND LESKOVEC, J. Defining and evaluating network communities based on ground-truth. In ICDM (2012), IEEE Computer Society, pp. 745–754.
    Google ScholarLocate open access versionFindings
  • YANG, R., XIAO, X., WEI, Z., BHOWMICK, S. S., ZHAO, J., AND LI, R. Efficient estimation of heat kernel pagerank for local clustering. In SIGMOD Conference (2019), ACM, pp. 1339– 1356.
    Google ScholarLocate open access versionFindings
  • YING, R., HE, R., CHEN, K., EKSOMBATCHAI, P., HAMILTON, W. L., AND LESKOVEC, J. Graph convolutional neural networks for web-scale recommender systems. In KDD (2018), ACM, pp. 974–983.
    Google ScholarLocate open access versionFindings
  • ZENG, H., ZHOU, H., SRIVASTAVA, A., KANNAN, R., AND PRASANNA, V. GraphSAINT: Graph sampling based inductive learning method. In ICLR (2020).
    Google ScholarLocate open access versionFindings
  • ZHAO, L., AND AKOGLU, L. Pairnorm: Tackling oversmoothing in gnns. In ICLR (2020).
    Google ScholarLocate open access versionFindings
  • ZHAO, L., PENG, X., TIAN, Y., KAPADIA, M., AND METAXAS, D. N. Semantic graph convolutional networks for 3d human pose regression. In CVPR (2019), pp. 3425–3435.
    Google ScholarLocate open access versionFindings
  • ZOU, D., HU, Z., WANG, Y., JIANG, S., SUN, Y., AND GU, Q. Layer-dependent importance sampling for training deep and large graph convolutional networks. In NeurIPS (2019), pp. 11247–11256. (2) If t =, we have R( ) is decremented by R( )(u, k) · Iuk and Q( ) is incremented by R( )(u, k) · Iuk. Consequently, we have
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
小科