An Attention-based Collaboration Framework for Multi-View Network Representation Learning

CIKM, pp. 1767-1776, 2017.

Cited by: 67|Bibtex|Views62|Links
EI
Keywords:
airline networkdata miningprogramming languagenetwork representationsocial networkMore(15+)
Weibo:
We studied learning node representations for networks with multiple views

Abstract:

Learning distributed node representations in networks has been attracting increasing attention recently due to its effectiveness in a variety of applications. Existing approaches usually study networks with a single type of proximity between nodes, which defines a single view of a network. However, in reality there usually exists multiple...More

Code:

Data:

0
Introduction
  • Mining and analyzing large-scale information networks has a racted a lot of a ention recently due to their wide applications in the real world.
  • There is a growing interest in representing networks into low-dimensional spaces (a.k.a, network embedding) [10, 20, 26], where each node is represented with a low-dimensional vector
  • Such vector representations are able to preserve the proximities between nodes, which can be treated as features and bene t a variety of downstream applications, such as node classi cation [20, 26], link prediction [10] and node visualization [26].
Highlights
  • Mining and analyzing large-scale information networks has a racted a lot of a ention recently due to their wide applications in the real world
  • 3.1 Collaboration Framework e goal of the collaboration framework is to capture the node proximities encoded in individual views and integrate them to vote for the robust node representations. erefore, for each node i , we introduce a set of view-speci c representations {xki }kK=1 to preserve the structure information encoded in individual views
  • We compare the robust node representations learned by MVE, node2vec-merge and MVE-NoCollab, and we report the performances on di erent node groups. e results are presented in Figure 3. e le groups contain nodes with larger degrees, in which the data are quite dense; while the right groups contain nodes with smaller degrees, and the data are very sparse
  • We studied learning node representations for networks with multiple views
  • We proposed an e ective framework to let di erent views collaborate with each other and vote for the robust node representations across di erent views
  • We evaluated the performance of our proposed approach on ve real-world networks with multiple views
Results
  • Experimental results on real-world networks show that the proposed approach outperforms existing state-of-the-art approaches for network representation learning with a single view and other competitive approaches with multiple views.
  • The authors conduct extensive experiments on various real world multiview networks.
  • Experimental results on both the multi-label node classi cation task and link prediction task show that the proposed approach outperforms state-of-the-art approaches for learning node representation with individual views and other competitive approaches with multiple views.
  • Comparing the running time of MVE and MVE-NoA n, the authors observe that the weight learning process in MVE takes less than 15% of the total running time on both datasets, which shows the good e ciency of the a ention based approach for weight learning
Conclusion
  • The authors studied learning node representations for networks with multiple views.
  • The authors evaluated the performance of the proposed approach on ve real-world networks with multiple views.
  • Experimental results on both the node classi cation task and link prediction task demonstrated the e ectiveness and e ciency of the proposed framework.
  • One promising direction is learning node representations for heterogeneous information networks, i.e., networks with multiple types of nodes and edges
  • In such networks, each meta-path [23] characterizes a type of proximity between the nodes, and various meta-paths yield networks with multiple views
Summary
  • Introduction:

    Mining and analyzing large-scale information networks has a racted a lot of a ention recently due to their wide applications in the real world.
  • There is a growing interest in representing networks into low-dimensional spaces (a.k.a, network embedding) [10, 20, 26], where each node is represented with a low-dimensional vector
  • Such vector representations are able to preserve the proximities between nodes, which can be treated as features and bene t a variety of downstream applications, such as node classi cation [20, 26], link prediction [10] and node visualization [26].
  • Results:

    Experimental results on real-world networks show that the proposed approach outperforms existing state-of-the-art approaches for network representation learning with a single view and other competitive approaches with multiple views.
  • The authors conduct extensive experiments on various real world multiview networks.
  • Experimental results on both the multi-label node classi cation task and link prediction task show that the proposed approach outperforms state-of-the-art approaches for learning node representation with individual views and other competitive approaches with multiple views.
  • Comparing the running time of MVE and MVE-NoA n, the authors observe that the weight learning process in MVE takes less than 15% of the total running time on both datasets, which shows the good e ciency of the a ention based approach for weight learning
  • Conclusion:

    The authors studied learning node representations for networks with multiple views.
  • The authors evaluated the performance of the proposed approach on ve real-world networks with multiple views.
  • Experimental results on both the node classi cation task and link prediction task demonstrated the e ectiveness and e ciency of the proposed framework.
  • One promising direction is learning node representations for heterogeneous information networks, i.e., networks with multiple types of nodes and edges
  • In such networks, each meta-path [23] characterizes a type of proximity between the nodes, and various meta-paths yield networks with multiple views
Tables
  • Table1: Statistics of the datasets
  • Table2: Table 2
  • Table3: Table 3
  • Table4: E ciency study. Our approach has close running time to LINE and node2vec. Learning weights of views takes less than 15% of the running time on both datasets
  • Table5: Examples of nearest neighbors according to similarity calculated by view-speci c node representations and robust node representations on the DBLP dataset
Download tables as Excel
Related work
  • Our work is related to the existing scalable approaches for learning network representations including DeepWalk [20], LINE [26] and node2vec [10], which use di erent search strategies to exploit the network structures: depth- rst search, breadth- rst search, and a

    Algorithm LINE node2vec MVE-NoA n

    DBLP 91.45 s 144.77 s 105.05 s 120.38 s

    Twitter 589.29 s 981.96 s 732.26 s 847.65 s combination of the two strategies. However, all these approaches focus on learning node representations for networks with a single view while we study networks with multiple views.

    e other line of the related work is multi-view learning, which aims to exploit information from multiple views and has shown e ectiveness in various tasks such as classi cation [2, 12, 29], clustering [3, 12, 12, 33, 37], ranking [34], topic modeling [28] and activity recovery [36]. e work which is the most similar to ours is the multi-view clustering [3, 12, 33, 37] and multi-view matrix factorization [9, 16, 22] methods. For example, Kumar et al [12] proposed a spectral clustering framework to regularize the clustering hypotheses across di erent views. Liu et al [33] proposed a multi-view nonnegative matrix factorization model, which aims to minimize the distance between the coe cient matrix of each view and the consensus matrix. Our multi-view network representation approach shares similar intuition with these pieces of work, aiming to nd robust data representations across multiple views. However, a major di erence is that existing approaches assign equal weights to all views, while our approach adopts an a ention based method, which learns di erent voting weights of views for di erent nodes.
Funding
  • Research was sponsored in part by the U.S Army Research Lab. under Cooperative Agreement No W911NF-09-2-0053 (NSCTA), National Science Foundation IIS-1320617 and IIS 16-18481, and grant 1U54GM114838 awarded by NIGMS through funds provided by the trans-NIH Big Data to Knowledge (BD2K) initiative (www.bd2k.nih.gov). e views and conclusions contained in this document are those of the author(s) and should not be interpreted as representing the o cial policies of the U.S Army Research Laboratory or the U.S. Government. e U.S Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon
  • Research was partially supported by the National Natural Science Foundation of China (NSFC Grant Nos. 61472006, 61772039 and 91646202)
Reference
  • D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.
    Findings
  • A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Computational learning theory, pages 92–100. ACM, 1998.
    Google ScholarLocate open access versionFindings
  • K. Chaudhuri, S. M. Kakade, K. Livescu, and K. Sridharan. Multi-view clustering via canonical correlation analysis. In Proceedings of the 26th annual international conference on machine learning, pages 129–136. ACM, 2009.
    Google ScholarLocate open access versionFindings
  • J. Chorowski, D. Bahdanau, K. Cho, and Y. Bengio. End-to-end continuous speech recognition using a ention-based recurrent nn: First results. arXiv preprint arXiv:1412.1602, 2014.
    Findings
  • M. De Domenico, A. Lima, P. Mougel, and M. Musolesi. e anatomy of a scienti c rumor. Scienti c reports, 3, 2013.
    Google ScholarFindings
  • R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. Liblinear: A library for large linear classi cation. e Journal of Machine Learning Research, 9:1871–1874, 2008.
    Google ScholarLocate open access versionFindings
  • T. Fawce. An introduction to roc analysis. Pa ern recognition le ers, 27(8):861– 874, 2006.
    Google ScholarLocate open access versionFindings
  • A. Franceschini, D. Szklarczyk, S. Frankild, M. Kuhn, M. Simonovic, A. Roth, J. Lin, P. Minguez, P. Bork, C. Von Mering, et al. String v9. 1: protein-protein interaction networks, with increased coverage and integration. Nucleic acids research, 41(D1):D808–D815, 2013.
    Google ScholarLocate open access versionFindings
  • D. Greene and P. Cunningham. A matrix factorization approach for integrating multiple data views. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 423–438.
    Google ScholarLocate open access versionFindings
  • A. Grover and J. Leskovec. node2vec: Scalable feature learning for networks.
    Google ScholarFindings
  • P. Jaillet, G. Song, and G. Yu. Airline network design and hub location problems. Location science, 4(3):195–212, 1996.
    Google ScholarLocate open access versionFindings
  • A. Kumar, P. Rai, and H. Daume. Co-regularized multi-view spectral clustering. In Advances in Neural Information Processing Systems, pages 1413–1421, 2011.
    Google ScholarLocate open access versionFindings
  • O. Levy and Y. Goldberg. Neural word embedding as implicit matrix factorization. In Advances in neural information processing systems, pages 2177–2185, 2014.
    Google ScholarLocate open access versionFindings
  • D. Liben-Nowell and J. Kleinberg. e link-prediction problem for social networks. Journal of the American society for information science and technology, 58(7):1019–1031, 2007.
    Google ScholarLocate open access versionFindings
  • A. Liberzon, A. Subramanian, R. Pinchback, H. orvaldsdoir, P. Tamayo, and J. P. Mesirov. Molecular signatures database (msigdb) 3.0. Bioinformatics, 27(12):1739–1740, 2011.
    Google ScholarLocate open access versionFindings
  • J. Liu, C. Wang, J. Gao, and J. Han. Multi-view clustering via joint nonnegative matrix factorization. In Proc. of SDM, volume 13, pages 252–260. SIAM, 2013.
    Google ScholarLocate open access versionFindings
  • T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119, 2013.
    Google ScholarLocate open access versionFindings
  • A. Mnih and Y. W. Teh. A fast and simple algorithm for training neural probabilistic language models. arXiv preprint arXiv:1206.6426, 2012.
    Findings
  • V. Mnih, N. Heess, A. Graves, et al. Recurrent models of visual a ention. In Advances in neural information processing systems, pages 2204–2212, 2014.
    Google ScholarLocate open access versionFindings
  • B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. arXiv preprint arXiv:1403.6652, 2014.
    Findings
  • D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Cognitive modeling, 5(3):1.
    Google ScholarLocate open access versionFindings
  • A. P. Singh and G. J. Gordon. Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 650–658. ACM, 2008.
    Google ScholarLocate open access versionFindings
  • Y. Sun, J. Han, X. Yan, P. S. Yu, and T. Wu. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment, 4(11):992–1003, 2011.
    Google ScholarLocate open access versionFindings
  • Y. Sun, Y. Yu, and J. Han. Ranking-based clustering of heterogeneous information networks with star network schema. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 797–806. ACM, 2009.
    Google ScholarLocate open access versionFindings
  • J. Tang, M., and Q. Mei. Pte: Predictive text embedding through largescale heterogeneous text networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1165– 1174. ACM, 2015.
    Google ScholarLocate open access versionFindings
  • J. Tang, M., M. Wang, M. Zhang, J. Yan, and Q. Mei. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, pages 1067–1077. ACM, 2015.
    Google ScholarLocate open access versionFindings
  • J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. Arnetminer: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 990–998. ACM, 2008.
    Google ScholarLocate open access versionFindings
  • J. Tang, M. Zhang, and Q. Mei. One theme in all views: modeling consensus topics in multiple contexts. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 5–13. ACM, 2013.
    Google ScholarLocate open access versionFindings
  • W. Wang and Z.-H. Zhou. A new analysis of co-training. In Proceedings of the 27th international conference on machine learning (ICML-10), pages 1135–1142, 2010.
    Google ScholarLocate open access versionFindings
  • X. Wang, L. Tang, H. Liu, and L. Wang. Learning with multi-resolution overlapping communities. Knowledge and Information Systems (KAIS), 2012.
    Google ScholarLocate open access versionFindings
  • S. Wasserman and K. Faust. Social network analysis: Methods and applications, volume 8. Cambridge university press, 1994.
    Google ScholarLocate open access versionFindings
  • S. J. Wright. Coordinate descent algorithms. Mathematical Programming, 151(1):3–34, 2015.
    Google ScholarLocate open access versionFindings
  • T. Xia, D. Tao, T. Mei, and Y. Zhang. Multiview spectral embedding. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 40(6):1438–1446, 2010.
    Google ScholarLocate open access versionFindings
  • J. Yu, Y. Rui, and B. Chen. Exploiting click constraints and multi-view features for image re-ranking. IEEE Transactions on Multimedia, 16(1):159–168, 2014.
    Google ScholarLocate open access versionFindings
  • R. Zafarani and H. Liu. Social computing data repository at ASU, 2009.
    Google ScholarFindings
  • C. Zhang, K. Zhang, Q. Yuan, H. Peng, Y. Zheng, T. Hanra y, S. Wang, and J. Han. Regions, periods, activities: Uncovering urban dynamics via cross-modal representation learning. In Proceedings of the 26th International Conference on World Wide Web, pages 361–370. International World Wide Web Conferences Steering Commi ee, 2017.
    Google ScholarLocate open access versionFindings
  • D. Zhou and C. J. Burges. Spectral clustering and transductive learning with multiple views. In Proceedings of the 24th international conference on Machine learning, pages 1159–1166. ACM, 2007.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments