AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
In this paper we propose a Structural Deep Network Embedding method, namely SDNE

Structural Deep Network Embedding

KDD, pp.1225-1234, (2016)

被引用1522|浏览309
EI
下载 PDF 全文
引用
微博一下

摘要

Network embedding is an important method to learn low-dimensional representations of vertexes in networks, aiming to capture and preserve the network structure. Almost all the existing network embedding methods adopt shallow models. However, since the underlying network structure is complex, shallow models cannot capture the highly non-li...更多

代码

数据

0
简介
  • Networks are ubiquitous and many real-world applications need to mine the information within these networks.
  • 15.00 DOI: http://dx.doi.org/10.1145/2939672.2939753 ment targeting often needs to cluster the users into communities in the social network.
  • An effective way is to embed networks into a low-dimensional space, i.e. learn vector representations for each vertex, with the goal of reconstructing the network in the learned embedding space.
  • As a result, mining information in networks, such as information retrieval [34], classification [15], and clustering [20], can be directly conducted in the low-dimensional space
重点内容
  • Nowadays, networks are ubiquitous and many real-world applications need to mine the information within these networks
  • With the first-order and second-order proximity, we can well characterize the local and global network structure, respectively. To preserve both the local and global network structure in the deep model, we propose a semi-supervised architecture, in which the unsupervised component reconstructs the second-order proximity to preserve the global network structure while the supervised component exploits the first-order proximity as the supervised information to preserve the local network structure
  • We report the results of the generalization of the network representations generated by different embedding methods on three classic data mining and machine learning applications, i.e. multilabel classification, link prediction and visualization
  • Before proceeding to evaluate the generalization of the proposed method on real-world applications, we first provide a basic evaluation on different network embedding methods with respect to their capability of network reconstruction. The reason for this experiment is that a good network embedding method should ensure that the the learned embeddings can preserve the original network structure
  • The result on Mean Average Precision is shown in Table 4
方法
  • The authors evaluate the proposed method on several realworld datasets and applications.
  • In order to comprehensively evaluate the effectiveness of the representations, the authors use five networked datasets, including three social networks, one citation network and one language network, for three real-world applications, i.e. multi-label classification, link prediction and visualization.
  • There are overall 39 different categories for BLOGCATALOG, 195 categories for FLICKR and 47 categories for categories
  • These categories can be used as the ground-truth of each vertex.
  • They can be evaluated on the multi-label classification task
结果
  • Before proceeding to evaluate the generalization of the proposed method on real-world applications, the authors first provide a basic evaluation on different network embedding methods with respect to their capability of network reconstruction.
  • The reason for this experiment is that a good network embedding method should ensure that the the learned embeddings can preserve the original network structure.
结论
  • The authors propose a Structural Deep Network Embedding, namely SDNE, to perform network embedding.
  • To capture the highly non-linear network structure, the authors design a semi-supervised deep model, which has multiple layers of nonlinear functions.
  • To further address the structure-preserving and sparsity problem, the authors jointly exploit the first-order proximity and second-order proximity to characterize the local and global network structure.
  • By jointly optimizing them in the semi-supervised deep model, the learned representations are local-global structurepreserved and are robust to sparse networks.
  • The results demonstrate substantial gains of the method compared with state-of-the-art
表格
  • Table1: Terms and Notations
  • Table2: Statistics of the dataset
  • Table3: Neural Network Structures
  • Table4: MAP on ARXIV-GRQC and BLOGCATALOG on reconstruction task
  • Table5: Table 5
  • Table6: KL-divergence for the 20-NEWSGROUP dataset
Download tables as Excel
相关工作
  • 2.1 Deep Neural Network

    Representation learning has long been an important problem of machine learning and many works aim at learning representations for samples [3, 35]. Recent advances in deep neural networks have witnessed that they have powerful representations abilities [12] and can generate very useful representations for many types of data. For example, [15] proposed a seven-layer convolutional neural network to generate image representations for classification. [33] proposed a multimodal deep model to learn image-text unified representations to achieve cross-modality retrieval task.

    However, to the best of our knowledge, there have been few deep learning works handling networks, especially learning network representations. In [9], Restricted Boltzmann Machines were adopted to do collaborative filtering. [30] adopted deep autoencoder to do graph clustering. [5] proposed a heterogeneous deep model to do heterogeneous data embedding. We differ from these works in two aspects. Firstly, the goals are different. Our work focuses on learning low-dimensional structure-preserved network representations which can be utilized among tasks. Secondly, we consider both the first-order and second-order proximity between vertexes to preserve the local and global network structure. But they only focus on one-order information.
基金
  • This work was supported by National Program on Key Basic Research Project, No 2015CB352300; National Natural Science Foundation of China, No 61370022, No 61531006, No 61472444 and No 61210008
引用论文
  • M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, 15(6):1373–1396, 2003.
    Google ScholarLocate open access versionFindings
  • Y. Bengio. Learning deep architectures for ai. Foundations and trends R in Machine Learning, 2(1):1–127, 2009.
    Google ScholarLocate open access versionFindings
  • Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(8):1798–1828, 2013.
    Google ScholarLocate open access versionFindings
  • S. Cao, W. Lu, and Q. Xu. Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pages 891–900. ACM, 2015.
    Google ScholarLocate open access versionFindings
  • S. Chang, W. Han, J. Tang, G.-J. Qi, C. C. Aggarwal, and T. S. Huang. Heterogeneous network embedding via deep architectures. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 119–128. ACM, 2015.
    Google ScholarLocate open access versionFindings
  • N. S. Dash. Context and contextual word meaning. SKASE Journal of Theoretical Linguistics, 5(2):21–31, 2008.
    Google ScholarLocate open access versionFindings
  • D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio. Why does unsupervised pre-training help deep learning? The Journal of Machine Learning Research, 11:625–660, 2010.
    Google ScholarLocate open access versionFindings
  • R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. Liblinear: A library for large linear classification. The Journal of Machine Learning Research, 9:1871–1874, 2008.
    Google ScholarLocate open access versionFindings
  • K. Georgiev and P. Nakov. A non-iid framework for collaborative filtering with restricted boltzmann machines. In ICML-13, pages 1148–1156, 2013.
    Google ScholarLocate open access versionFindings
  • G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. Signal Processing Magazine, IEEE, 29(6):82–97, 2012.
    Google ScholarLocate open access versionFindings
  • G. E. Hinton, S. Osindero, and Y.-W. Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527–1554, 2006.
    Google ScholarLocate open access versionFindings
  • G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 2006.
    Google ScholarLocate open access versionFindings
  • M. Jamali and M. Ester. A matrix factorization technique with trust propagation for recommendation in social networks. In Proceedings of the fourth ACM conference on Recommender systems, pages 135–142. ACM, 2010.
    Google ScholarLocate open access versionFindings
  • E. M. Jin, M. Girvan, and M. E. Newman. Structure of growing social networks. Physical review E, 64(4):046132, 2001.
    Google ScholarLocate open access versionFindings
  • A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
    Google ScholarLocate open access versionFindings
  • J. Leskovec, J. Kleinberg, and C. Faloutsos. Graph evolution: Densification and shrinking diameters. ACM Transactions on Knowledge Discovery from Data (TKDD), 1(1):2, 2007.
    Google ScholarLocate open access versionFindings
  • D. Liben-Nowell and J. Kleinberg. The link-prediction problem for social networks. Journal of the American society for information science and technology, 58(7):1019–1031, 2007.
    Google ScholarLocate open access versionFindings
  • T. Liu and D. Tao. Classification with noisy labels by importance reweighting. TPAMI, (1):1–1.
    Google ScholarFindings
  • D. Luo, F. Nie, H. Huang, and C. H. Ding. Cauchy graph embedding. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 553–560, 2011.
    Google ScholarLocate open access versionFindings
  • A. Y. Ng, M. I. Jordan, Y. Weiss, et al. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 2:849–856, 2002.
    Google ScholarLocate open access versionFindings
  • B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. In SIGKDD, pages 701–710. ACM, 2014.
    Google ScholarLocate open access versionFindings
  • S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000.
    Google ScholarLocate open access versionFindings
  • R. Salakhutdinov and G. Hinton. Semantic hashing. International Journal of Approximate Reasoning, 50(7):969–978, 2009.
    Google ScholarLocate open access versionFindings
  • B. Shaw and T. Jebara. Structure preserving embedding. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 937–944. ACM, 2009.
    Google ScholarLocate open access versionFindings
  • R. Socher, A. Perelygin, J. Y. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP), volume 1631, page 1642.
    Google ScholarLocate open access versionFindings
  • J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, pages 1067–1077. International World Wide Web Conferences Steering Committee, 2015.
    Google ScholarLocate open access versionFindings
  • L. Tang and H. Liu. Relational learning via latent social dimensions. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2009.
    Google ScholarLocate open access versionFindings
  • L. Tang and H. Liu. Scalable learning of collective behavior based on sparse social dimensions. In Proceedings of the 18th ACM conference on Information and knowledge management, pages 1107–1116. ACM, 2009.
    Google ScholarLocate open access versionFindings
  • J. B. Tenenbaum, V. De Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, 2000.
    Google ScholarLocate open access versionFindings
  • F. Tian, B. Gao, Q. Cui, E. Chen, and T.-Y. Liu. Learning deep representations for graph clustering. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pages 1293–1299, 2014.
    Google ScholarLocate open access versionFindings
  • L. Van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(2579-2605):85, 2008.
    Google ScholarLocate open access versionFindings
  • S. V. N. Vishwanathan, N. N. Schraudolph, R. Kondor, and K. M. Borgwardt. Graph kernels. The Journal of Machine Learning Research, 11:1201–1242, 2010.
    Google ScholarLocate open access versionFindings
  • D. Wang, P. Cui, M. Ou, and W. Zhu. Deep multimodal hashing with orthogonal regularization. In Proceedings of the 24th International Conference on Artificial Intelligence, pages 2291–2297. AAAI Press, 2015.
    Google ScholarLocate open access versionFindings
  • Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In Advances in neural information processing systems, pages 1753–1760, 2009.
    Google ScholarLocate open access versionFindings
  • C. Xu, D. Tao, and C. Xu. Multi-view intact space learning. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 37(12):2531–2544, 2015.
    Google ScholarLocate open access versionFindings
  • J. Zhuang, I. W. Tsang, and S. Hoi. Two-layer multiple kernel learning. In International conference on artificial intelligence and statistics, pages 909–917, 2011.
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科