AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
We propose a sequential deep matching model to capture users’ dynamic preferences by combining short-term sessions and long-term behaviors

SDM: Sequential Deep Matching Model for Online Large-scale Recommender System

Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp.2635-2643, (2019)

被引用4|浏览35
EI
下载 PDF 全文
引用
微博一下

摘要

Capturing users' precise preferences is a fundamental problem in large-scale recommender system. Currently, item-based Collaborative Filtering (CF) methods are common matching approaches in industry. However, they are not effective to model dynamic and evolving preferences of users. In this paper, we propose a new sequential deep matching...更多

代码

数据

0
简介
  • Large-scale recommender systems in industry are required to have both accurate prediction of users’ preferences and quick response to their current need.
  • Online deployed matching models at Taobao are mainly based on item-based Collaborative Filtering (CF) methods [16, 22].
  • They model static useritem interactions and do not well capture dynamic transformation in users’ whole behavior sequences.
  • Such methods usually lead to homogeneous recommendation.
  • To accurately understand interests and preferences of users, sequential order information should be incorporated into the matching module
重点内容
  • Large-scale recommender systems in industry are required to have both accurate prediction of users’ preferences and quick response to their current need
  • We develop a novel sequential deep matching (SDM) model for large-scale recommender system in real-world applications by considering both short- and long-term behaviors
  • Average pooling over item sequences neglects the inherit correlation among items causing hurts on recommending quality (Recall, Precision) severely
  • GRU4REC and NARM consider the evolution of short-term behaviors
  • We propose a sequential deep matching model to capture users’ dynamic preferences by combining short-term sessions and long-term behaviors
  • Extensive offline experiments show the effectiveness of our model
方法
  • The authors use the following methods to compare with the model on two offline datasets. The authors include five variants of the model for ablation study.

    Item-based CF [16].
  • The authors use the following methods to compare with the model on two offline datasets.
  • The authors include five variants of the model for ablation study.
  • Item-based CF [16].
  • It’s one of the major candidate generation approaches in industry.
  • Collaborative Filtering method generates item-item similarity matrix for recommending.
  • DNN [4].
  • A deep neural network based recommendation approach proposed by YouTube.
  • Vectors of videos and users are concatenated and fed into a multi-layer feed forward neural network
结果
  • Results on offline datasets of different models are shown in Table 2.
  • The authors select the best results from all the training epochs of these models.
  • GRU4REC and NARM consider the evolution of short-term behaviors.
  • They perform better than original DNN models.
  • The reason why SHAN and BINN can beat GRU4Rec in almost all metrics is that they incorporate more personalized information including long-term behaviors and user profile representation
结论
  • The authors propose a sequential deep matching model to capture users’ dynamic preferences by combining short-term sessions and long-term behaviors.
  • The authors employ multi-head self-attention to capture multiple interests in short-term sessions and long-short term gated fusion network to incorporate long-term preferences.
  • Extensive offline experiments show the effectiveness of the model.
  • The matching model is successfully deployed on Taobao’s recommender system with improvements in term of important commercial metrics
总结
  • Introduction:

    Large-scale recommender systems in industry are required to have both accurate prediction of users’ preferences and quick response to their current need.
  • Online deployed matching models at Taobao are mainly based on item-based Collaborative Filtering (CF) methods [16, 22].
  • They model static useritem interactions and do not well capture dynamic transformation in users’ whole behavior sequences.
  • Such methods usually lead to homogeneous recommendation.
  • To accurately understand interests and preferences of users, sequential order information should be incorporated into the matching module
  • Objectives:

    The authors' goal is to recall top N items after the user sequences as matching candidates. The authors' goal is to predict top N item candidates at time t + 1 based on the scores of inner product between out and each column vector vi in V.
  • Methods:

    The authors use the following methods to compare with the model on two offline datasets. The authors include five variants of the model for ablation study.

    Item-based CF [16].
  • The authors use the following methods to compare with the model on two offline datasets.
  • The authors include five variants of the model for ablation study.
  • Item-based CF [16].
  • It’s one of the major candidate generation approaches in industry.
  • Collaborative Filtering method generates item-item similarity matrix for recommending.
  • DNN [4].
  • A deep neural network based recommendation approach proposed by YouTube.
  • Vectors of videos and users are concatenated and fed into a multi-layer feed forward neural network
  • Results:

    Results on offline datasets of different models are shown in Table 2.
  • The authors select the best results from all the training epochs of these models.
  • GRU4REC and NARM consider the evolution of short-term behaviors.
  • They perform better than original DNN models.
  • The reason why SHAN and BINN can beat GRU4Rec in almost all metrics is that they incorporate more personalized information including long-term behaviors and user profile representation
  • Conclusion:

    The authors propose a sequential deep matching model to capture users’ dynamic preferences by combining short-term sessions and long-term behaviors.
  • The authors employ multi-head self-attention to capture multiple interests in short-term sessions and long-short term gated fusion network to incorporate long-term preferences.
  • Extensive offline experiments show the effectiveness of the model.
  • The matching model is successfully deployed on Taobao’s recommender system with improvements in term of important commercial metrics
表格
  • Table1: Statistics of offline and online datasets
  • Table2: Comparisons of different models on offline datasets of Taobao and JD
  • Table3: Results of various number of heads. (K = 100)
  • Table4: Comparisons of different fusion methods. (K = 100)
Download tables as Excel
相关工作
  • 2.1 Deep Matching in Industry

    To develop more effective matching models in industrial recommender system, many researchers adopt deep neural networks which have the powerful representation ability. Models based on Matrix Factorization (MF) [13] try to decompose pairwise user-item implicit feedback into user and item vectors. YouTube [4] uses deep neural network to learn both embeddings of users and items. The two kinds of embeddings are generated from their corresponding features separately. The prediction is made as equivalent to search the nearest neighbors of users’ vectors among all the items. Besides, Zhu et al [34] proposes a novel tree-based large-scale recommender system, which can provide novel items and overcome the calculation barrier of vector search. Recently, graph embedding based methods are applied in many industrial applications to complement or replace traditional methods. Wang et al [26] proposes to construct an item graph from users’ behavior history and then applies the state-of-the-art graph embedding methods to learn the embedding of each item. To address the cold start and sparsity problem, they incorporate side information of items to enhance the embedding procedure. Ying et al [30] develops and deploys an effective and efficient graph convolutional network at Pinterest2 to generate embeddings of nodes (items) that incorporates both graph structure as well as node feature information. But these models can’t well take the dynamic and evolving of users’ preferences into consideration. In this work, we consider this in matching stage by introducing sequential recommendation.
研究对象与分析
thousands active users: 10
During training process, we remove sessions whose length are less than 2. In the test stage, we select approximately 10 thousands active users in the 8th day for quick evaluation. Their first 25%

GPU workers with average 450 global steps: 100
To be fair, all of these models share the same training and testing datasets, as well as input features of items and users and other training hyper-parameters. For training, we use 5 parameter severs (PSs) and 6 GPU (Tesla P100-PCIE-16GB) workers with average 30 global steps/s on offline experiment and we use 20 PSs and 100 GPU workers with average 450 global steps/s on online experiment. Adam optimizer with learning rate 0.001 is used to update parameters and gradient clipping is adopted by scaling gradients when the norm exceeded a threshold of 5

引用论文
  • Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).
    Findings
  • Ting Bai, Pan Du, Wayne Xin Zhao, Ji-Rong Wen, and Jian-Yun Nie. 2019. A LongShort Demands-Aware Model for Next-Item Recommendation. arXiv preprint arXiv:1903.00066 (2019).
    Findings
  • Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2018. Sequential recommendation with user memory networks. In WSDM. ACM, 108–116.
    Google ScholarLocate open access versionFindings
  • Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In RecSys. ACM, 191–198.
    Google ScholarLocate open access versionFindings
  • Disheng Dong, Xiaolin Zheng, Ruixun Zhang, and Yan Wang. 2018. Recurrent Collaborative Filtering for Unifying General and Sequential Recommender. In IJCAI. 3350–3356.
    Google ScholarFindings
  • Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, and Keping Yang. 2019. Deep Session Interest Network for Click-Through Rate Prediction. In
    Google ScholarLocate open access versionFindings
  • Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk.
    Google ScholarFindings
  • 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
    Findings
  • [8] Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji-Rong Wen, and Daxiang Dong. 201Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In WSDM. ACM, 573–581.
    Google ScholarLocate open access versionFindings
  • [9] Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. 2018. Improving sequential recommendation with knowledge-enhanced memory networks. In SIGIR. ACM, 505–514.
    Google ScholarLocate open access versionFindings
  • [10] Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2014. On using very large target vocabulary for neural machine translation. arXiv preprint arXiv:1412.2007 (2014).
    Findings
  • [11] Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with GPUs. arXiv preprint arXiv:1702.08734 (2017).
    Findings
  • [12] Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDE. IEEE, 197–206.
    Google ScholarLocate open access versionFindings
  • [13] Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 8 (2009), 30–37.
    Google ScholarLocate open access versionFindings
  • [14] Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In CIKM. ACM, 1419–1428.
    Google ScholarLocate open access versionFindings
  • [15] Zhi Li, Hongke Zhao, Qi Liu, Zhenya Huang, Tao Mei, and Enhong Chen. 2018. Learning from history and present: next-item recommendation via discriminatively exploiting user behaviors. In KDD. ACM, 1734–1743.
    Google ScholarLocate open access versionFindings
  • [16] Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing 1 (2003), 76–80.
    Google ScholarLocate open access versionFindings
  • [17] Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. 20STAMP: shortterm attention/memory priority model for session-based recommendation. In KDD. ACM, 1831–1839.
    Google ScholarLocate open access versionFindings
  • [18] Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015).
    Findings
  • [19] Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2017. Regularizing and optimizing LSTM language models. arXiv preprint arXiv:1708.02182 (2017).
    Findings
  • [20] Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. 2017. Personalizing session-based recommendations with hierarchical recurrent neural networks. In RecSys. ACM, 130–137.
    Google ScholarLocate open access versionFindings
  • [21] Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW. ACM, 811–820.
    Google ScholarLocate open access versionFindings
  • [22] Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In WWW. ACM, 285–295.
    Google ScholarLocate open access versionFindings
  • [23] Jiaxi Tang, Francois Belletti, Sagar Jain, Minmin Chen, Alex Beutel, Can Xu, and Ed H Chi. 2019. Towards Neural Mixture Recommender for Long Range Dependent User Sequences. arXiv preprint arXiv:1902.08588 (2019).
    Findings
  • [24] Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM. ACM, 565–573.
    Google ScholarLocate open access versionFindings
  • [25] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. 5998–6008.
    Google ScholarFindings
  • [26] Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba. arXiv preprint arXiv:1803.02349 (2018).
    Findings
  • [27] Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, Shengxian Wan, and Xueqi Cheng. 2015. Learning hierarchical representation model for nextbasket recommendation. In SIGIR. ACM, 403–412.
    Google ScholarLocate open access versionFindings
  • [28] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016).
    Findings
  • [29] Haochao Ying, Fuzhen Zhuang, Fuzheng Zhang, Yanchi Liu, Guandong Xu, Xing Xie, Hui Xiong, and Jian Wu. 2018. Sequential Recommender System based on Hierarchical Attention Networks. In IJCAI.
    Google ScholarLocate open access versionFindings
  • [30] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. arXiv preprint arXiv:1806.01973 (2018).
    Findings
  • [31] Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2019. A Simple Convolutional Generative Network for Next Item Recommendation. In WSDM. ACM, 582–590.
    Google ScholarLocate open access versionFindings
  • [32] Shuai Zhang, Yi Tay, Lina Yao, and Aixin Sun. 2018. Next item recommendation with self-attention. arXiv preprint arXiv:1808.06414 (2018).
    Findings
  • [33] Wei Zhao, Benyou Wang, Jianbo Ye, Yongqiang Gao, Min Yang, and Xiaojun Chen. 2018. PLASTIC: Prioritize Long and Short-term Information in Top-n Recommendation using Adversarial Training. In IJCAI. 3676–3682.
    Google ScholarFindings
  • [34] Han Zhu, Xiang Li, Pengye Zhang, Guozheng Li, Jie He, Han Li, and Kun Gai. 2018. Learning Tree-based Deep Model for Recommender Systems. In KDD. ACM, 1079–1088.
    Google ScholarLocate open access versionFindings
作者
Taiwei Jin
Taiwei Jin
Changlong Yu
Changlong Yu
Quan Lin
Quan Lin
Keping Yang
Keping Yang
您的评分 :
0

 

标签
评论
小科