Learning to Transfer Graph Embeddings for Inductive Graph based Recommendation

SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval Virtual Event China July, 2020, pp. 1211-1220, 2020.

Cited by: 0|Bibtex|Views144|DOI:https://doi.org/10.1145/3397271.3401145
EI
Other Links: arxiv.org|dl.acm.org|dblp.uni-trier.de|academic.microsoft.com
Weibo:
We further proposed a transfer network that learns to transform the item content embedding to the graph neural network space

Abstract:

With the increasing availability of videos, how to edit them and present the most interesting parts to users, i.e., video highlight, has become an urgent need with many broad applications. As users' visual preferences are subjective and vary from person to person, previous generalized video highlight extraction models fail to tailor to us...More

Code:

Data:

0
Introduction
  • With the increasing availability of camera devices, videos are ubiquitous on entertainment and social networking platforms
  • As these videos are usually unstructured and long-running, it is non-trivial to directly browse the interesting and representative parts and share these parts in the social media.
  • Editing such videos into the highlight segments, i.e., the most interesting or representative parts, and presenting these parts is a natural choice.
  • Instead of presenting the same highlight parts to all users, an ideal video highlight system should suggest and recommend personalized highlight parts to tailor to users’ personalized preferences
Highlights
  • With the increasing availability of camera devices, videos are ubiquitous on entertainment and social networking platforms
  • We propose a general framework: an inductive Graph based Transfer learning framework for personalized video highlight Recommendation (TransGRec)
  • In order to generalize to unseen nodes in test stage, we transfer the embeddings learned in the source network (i.e., Graph Neural Networks (GNN)) of the training data to learn an approximated item embedding in the target network
  • The graph neural network can be regarded as the complex teacher network, while the transfer network resembles a student network that distills the knowledge learned from the teacher network
  • We further proposed a transfer network that learns to transform the item content embedding to the graph neural network space
  • Though we use the personalized video recommendation as an application scenario, our proposed model is generally applicable to any content based recommendation tasks
Conclusion
  • The authors formulate the video highlight recommendation problem as an inductive representation learning on graph neural networks.
  • The graph neural network can be regarded as the complex teacher network, while the transfer network resembles a student network that distills the knowledge learned from the teacher network.In this paper, the authors designed a TransGRec framework for personalized video highlight recommendation.
  • The authors based TransGRec on an graph neural network model, and proposed to propagate user embeddings in the graph to alleviate the cold-start user problem.
  • The authors would like to apply and validate the effectiveness of the proposed framework in the general recommender systems
Summary
  • Introduction:

    With the increasing availability of camera devices, videos are ubiquitous on entertainment and social networking platforms
  • As these videos are usually unstructured and long-running, it is non-trivial to directly browse the interesting and representative parts and share these parts in the social media.
  • Editing such videos into the highlight segments, i.e., the most interesting or representative parts, and presenting these parts is a natural choice.
  • Instead of presenting the same highlight parts to all users, an ideal video highlight system should suggest and recommend personalized highlight parts to tailor to users’ personalized preferences
  • Objectives:

    The personalized video highlight recommendation task asks: with user-segment rating matrix R, for each user u to each test video v, the goal is to recommend Top-N ranking list of segments which meet each user’s personalized preference.
  • Conclusion:

    The authors formulate the video highlight recommendation problem as an inductive representation learning on graph neural networks.
  • The graph neural network can be regarded as the complex teacher network, while the transfer network resembles a student network that distills the knowledge learned from the teacher network.In this paper, the authors designed a TransGRec framework for personalized video highlight recommendation.
  • The authors based TransGRec on an graph neural network model, and proposed to propagate user embeddings in the graph to alleviate the cold-start user problem.
  • The authors would like to apply and validate the effectiveness of the proposed framework in the general recommender systems
Tables
  • Table1: The statistics of the dataset
  • Table2: Overall performance comparison ↑ means the larger value, the better performance; ↓ means the smaller value, the better performance)
  • Table3: Recommendation metrics comparisons of different Top-N values
  • Table4: Effects of different propagation layer depth K in TransGRec-E
  • Table5: Effects of different propagation layer depth K in TransGRec-A
Download tables as Excel
Related work
  • 5.1 Video Highlight and Personalization

    Automatic video highlight extraction and summarization deals with selecting representative segments from videos [5, 30]. These models learned interesting, representative, or diversified visual content segments based on well-defined optimization goals [5, 49]. As many users edited videos in online platforms, some researchers proposed to leverage the wisdom of the crowds as ground-truth or priors to guide video highlight extraction [11, 19, 30]. However, these proposed models neglected users’ personalized preferences. With the huge boom of video editing platforms and APPs, recently researchers published a personalized video highlight dataset that records each user’s liked segments of videos, and proposed a personalized highlight detection model [32]. The personalization is achieved by adapting the input of each user as an aggregation of her liked segments of videos, and the proposed personalization model showed better performance compared to the state-of-the-art generalized highlight extraction model [11], indicating the soundness of personalization in video highlight recommendation. However, as each user’s annotated records are limited, the recommendation performance is still far from satisfaction.
Funding
  • This work was supported in part by the National Natural Science Foundation of China(Grant No.61725203, 61972125, U1936219, 61722204, 61932009 and 61732008)
Reference
  • Geoffrey E. Hinton Alex Krizhevsky, Ilya Sutskever. 2012. Imagenet classification with deep convolutional neural networks. In NIPS. 1097–1105.
    Google ScholarFindings
  • Grigory Antipov, Moez Baccouche, and Jean-Luc Dugelay. 2017. Face aging with conditional generative adversarial networks. In ICIP. 2089–2093.
    Google ScholarFindings
  • Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. 2017. A generic coordinate descent framework for learning from implicit feedback. In WWW. 1341–1350.
    Google ScholarFindings
  • Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. JMLR 7, Nov (2006), 2399–2434.
    Google ScholarLocate open access versionFindings
  • Eric P. Xing Bin Zhao. 2014. Quasi real-time summarization for consumer videos. In CVPR. 2513–2520.
    Google ScholarFindings
  • Rose Catherine and William Cohen. 2017. Transnets: Learning to transform for recommendation. In Recsys. 288–296.
    Google ScholarFindings
  • Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In NIPS. 2172–2180.
    Google ScholarFindings
  • Rob Fergus Lorenzo Torresani Manohar Paluri Du Tran, Lubomir D. Bourdev. 2015. Learning spatiotemporal features with 3d convolutional networks. In ICCV. 4489–4497.
    Google ScholarFindings
  • Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In NIPS. 2672–2680.
    Google ScholarFindings
  • Michael Gygli. 2018. Ridiculously fast shot boundary detection with fully convolutional neural networks. In CBMI. 1–4.
    Google ScholarLocate open access versionFindings
  • Michael Gygli, Yale Song, and Liangliang Cao. 2016. Video2gif: Automatic generation of animated gifs from video. In CVPR. 1001–1009.
    Google ScholarFindings
  • Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NIPS. 1024–1034.
    Google ScholarFindings
  • Ruining He and Julian McAuley. 2016. VBPR: visual bayesian personalized ranking from implicit feedback. In AAAI. 144–150.
    Google ScholarLocate open access versionFindings
  • Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 20Distilling the knowledge in a neural network. In NIPS Workshop.
    Google ScholarLocate open access versionFindings
  • Kurt Hornik, Maxwell Stinchcombe, and Halbert White. 1989. Multilayer feedforward networks are universal approximators. Neural networks 2, 5 (1989), 359–366.
    Google ScholarLocate open access versionFindings
  • Min Hou, Le Wu, Enhong Chen, Zhi Li, Zheng Vincent W., and Qi Liu. 2019. Explainable Fashion Recommendation: A Semantic Attribute Region Guided Approach.. In IJCAI. 4681–4688.
    Google ScholarFindings
  • Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu. 2012. 3D convolutional neural networks for human action recognition. TPAMI 35, 1 (2012), 221–231.
    Google ScholarLocate open access versionFindings
  • Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. 2014. Large-scale video classification with convolutional neural networks. In CVPR. 1725–1732.
    Google ScholarFindings
  • Aditya Khosla, Raffay Hamid, Chih-Jen Lin, and Neel Sundaresan. 2013. Largescale video summarization using web-image priors. In CVPR. 2698–2705.
    Google ScholarFindings
  • Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.
    Google ScholarFindings
  • Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Predict then propagate: Graph neural networks meet personalized pagerank. In ICLR.
    Google ScholarFindings
  • Joonseok Lee and Sami Abu-El-Haija. 2017. Large-scale content-only video recommendation. In ICCV. 987–995.
    Google ScholarLocate open access versionFindings
  • Yong Jae Lee and Kristen Grauman. 2015. Predicting important objects for egocentric video summarization. IJCV 114, 1 (2015), 38–55.
    Google ScholarLocate open access versionFindings
  • Chenyi Lei, Dong Liu, Weiping Li, Zheng-Jun Zha, and Houqiang Li. 2016. Comparative deep learning of hybrid representations for image recommendations. In CVPR. 2545–2553.
    Google ScholarFindings
  • Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI. 3538–3545.
    Google ScholarFindings
  • Defu Lian, Qi Liu, and Enhong Chen. 2020. Personalized Ranking with Importance Sampling. In Proceedings of The Web Conference 2020. 1093–1103.
    Google ScholarLocate open access versionFindings
  • Defu Lian, Haoyu Wang, Zheng Liu, Jianxun Lian, Enhong Chen, and Xing Xie. 2020. LightRec: A Memory and Search-Efficient Recommender System. In Proceedings of The Web Conference 2020. 695–705.
    Google ScholarLocate open access versionFindings
  • David Lopez-Paz, Léon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. 2016. Unifying distillation and privileged information. In ICLR.
    Google ScholarFindings
  • Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. JMLR 9, Nov (2008), 2579–2605.
    Google ScholarLocate open access versionFindings
  • Luc Van Gool Michael Gygli, Helmut Grabner. 2015. Video summarization by learning submodular mixtures of objectives. In CVPR. 3090–3098.
    Google ScholarFindings
  • Steven M. Seitz Min Sun, Ali Farhadi. 2014. Ranking domain-specific highlights by analyzing edited videos. In ECCV. 787–802.
    Google ScholarLocate open access versionFindings
  • Ana Molino and Michael Gygli. 2018. PHD-GIFs: Personalized Highlight Detection for Automatic GIF Creation. In MM. 600–608.
    Google ScholarLocate open access versionFindings
  • Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. TKDE 22, 10 (2009), 1345–1359.
    Google ScholarLocate open access versionFindings
  • Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In UAI. 452– 461.
    Google ScholarFindings
  • Pablo Castells RocÃŋo CaÃśamares. 2018. From the PRP to the Low Prior Discovery Recall Principle for Recommender Systems. In SIGIR. 1081–1084.
    Google ScholarFindings
  • Peijie Sun, Le Wu, and Meng Wang. 2018. Attentive Recurrent Social Recommendation. In SIGIR. 185–194.
    Google ScholarLocate open access versionFindings
  • Peijie Sun, Le Wu, Kun Zhang, Yanjie Fu, Richang Hong, and Meng Wang. 2020. Dual Learning for Explainable Recommendation: Towards Unifying User Preference Prediction and Review Generation. In WWW. 837–847.
    Google ScholarLocate open access versionFindings
  • Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph Convolutional Matrix Completion. In ICLR.
    Google ScholarLocate open access versionFindings
  • Dieleman S. Van den Oord, A. and B. Schrauwen. 2013. Deep content-based music recommendation. In NIPS. 2643–2651.
    Google ScholarFindings
  • Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. 2017. Dropoutnet: Addressing cold start in recommender systems. In NIPS. 4957–4966.
    Google ScholarFindings
  • Wang N. Wang, H. and D. Y. Yeung. 2015. Collaborative deep learning for recommender systems. In KDD. 1235–1244.
    Google ScholarFindings
  • Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural Graph Collaborative Filtering. In SIGIR. 165–174.
    Google ScholarLocate open access versionFindings
  • Le Wu, Lei Chen, Richang Hong, Yanjie Fu, Xing Xie, and Meng Wang. 2019. A hierarchical attention model for social contextual image recommendation. TKDE (2019).
    Google ScholarLocate open access versionFindings
  • Le Wu, Lei Chen, Yonghui Yang, Richang Hong, Yong Ge, Xing Xie, and Meng Wang. 2019. Personalized Multimedia Item and Key Frame Recommendation. In IJCAI. 1431–1437.
    Google ScholarLocate open access versionFindings
  • Le Wu, Peijie Sun, Yanjie Fu, Richang Hong, Xiting Wang, and Meng Wang. 2019. A Neural Influence Diffusion Model for Social Recommendation. In SIGIR. 235–244.
    Google ScholarLocate open access versionFindings
  • Yuexin Wu, Hanxiao Liu, and Yiming Yang. 2018. Graph Convolutional Matrix Completion for Bipartite Edge Prediction. In KDIR. 51–60.
    Google ScholarLocate open access versionFindings
  • Ting Yao, Tao Mei, and Yong Rui. 2016. Highlight detection with pairwise deep ranking for first-person video summarization. In CVPR. 982–990.
    Google ScholarLocate open access versionFindings
  • Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. In KDD. 974–983.
    Google ScholarLocate open access versionFindings
  • Chao W. L. Sha F. Zhang, K. and K. Grauman. 2016. Video summarization with long short-term memory. In ECCV. 766–782.
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments