AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically

Go Generating

AI Traceability

AI parses the academic lineage of this thesis

Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper

We explore the dynamic meaning of items in realworld scenarios and propose a novel next-item recommendation framework empowered by sequential hypergraphs to incorporate the short-term item correlations for dynamic item embedding

Next-item Recommendation with Sequential Hypergraphs

SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Re..., pp.1101-1110, (2020)

Cited by: 10|Views285


There is an increasing attention on next-item recommendation systems to infer the dynamic user preferences with sequential user interactions. While the semantics of an item can change over time and across users, the item correlations defined by user interactions in the short term can be distilled to capture such change, and help in uncove...More



  • In online platforms with millions of available items and churn in new items, recommendation systems act as an essential component to connect users with interesting items.
  • The authors can infer that the bouquet purchased by User was for a wedding since she purchased items typically associated with weddings
  • To capture these changes in item semantics, the authors propose to model such short-term item correlations in a hypergraph [1, 11], in which each hyperedge can connect multiple nodes on a single edge.
  • While each node in the hypergraph denotes an item, a hyperedge can connect the set of items a user interacts with in the short time period altogether
  • In online platforms with millions of available items and churn in new items, recommendation systems act as an essential component to connect users with interesting items
  • To model the influence of item embeddings in the past time periods, we develop a residual gating layer to combine the dynamic item embeddings of the previous time period with the static item embeddings to generate the input for the hypergraph convolutional network (HGCN)
  • With extensive experiments on datasets covering different online platforms including ecommerce websites (Amazon and Etsy) and an information sharing community (Goodreads), the proposed model outperforms state-of-the-art models in providing Top-K next-item recommendation
  • We introduce the details of the proposed HyperRec, centered around three guiding research questions: RQ1 How to define correlations between items with a hypergraph structure and how to effectively incorporate the short-term item correlations into dynamic item embeddings by considering multi-hop connections between items? RQ2 While the meaning of items in the past can hint on their characteristics in the future, how to link the embedding process at different time periods to connect how the residual information flows between consecutive time periods? RQ3 How to fuse the short-term user intent with the dynamic item embedding to represent each interaction in a user interaction sequence for dynamic user preference modeling?
  • As an advanced version of GRU4Rec targeting Top-K recommendation, in Amazon and Goodreads, GRU4Rec+ can improve TCN and HPMN by conducting dynamic user modeling with Gated Recurrent Neural Network (GRU) and adopting a loss function tailored to RNN-based models for Top-K recommendation
  • We explore the dynamic meaning of items in realworld scenarios and propose a novel next-item recommendation framework empowered by sequential hypergraphs to incorporate the short-term item correlations for dynamic item embedding
  • The authors conduct experiments to evaluate the performance of the proposed HyperRec over datasets sampled from three online platforms (Goodreads, Amazon and Etsy).
  • To explore the generalization of the proposed model, the authors sample data from three different online platforms.
  • Summary statistics of these datasets are in Table 1
  • Evaluation Metrics

    Following the leave-one-out setting, in the test data, each user only relates to one item that the user interacts with after the cutting time.
  • The newly proposed HGN is equipped with a novel feature-gating and an instance gating to enhance the short-term user modeling, and can outperform the aforementioned baselines
  • Both SASRec and Bert4Rec employ a self-attention layer to model the sequential user patterns.
  • In BERT4Rec, by randomly masking items in the user sequences, it is able to train a bidirectional model for recommendation
  • It does not bring in huge improvement as in the original BERT applications for natural language processing since the right-to-left patterns in sequences are not necessarily informative for predicting dynamic user preferences
  • The authors explore the dynamic meaning of items in realworld scenarios and propose a novel next-item recommendation framework empowered by sequential hypergraphs to incorporate the short-term item correlations for dynamic item embedding.
  • With the stacking of hypergraph convolution networks, a residual gating and the fusion layer, the proposed model is able to provide more accurate modeling of user preferences, leading to improved performance compared to the state-of-the-art in predicting user’s action for both ecommerce (Amazon and Etsy) and information sharing platform (Goodreads).
  • The authors are interested in investigating how to transfer the dynamic patterns across platforms or across domains for an improved predictive performance
  • Table1: Statistics of the datasets
  • Table2: Comparison of Different Models. ∗ indicates that the improvement of the best result is statistically significant compared with the next-best result with < 0.01
  • Table3: Results for Ablation Test under HIT@1/NDCG@1
Download tables as Excel
Related work
  • Next-item Recommendation. Next-item recommendation has been a promising research topic recently. Compared with recommendation systems treating users as static, it usually updates a user’s status after each of her interactions and generates predictions relying on the relationships between items consumed sequentially. Some works focus on recommendation for short-term interaction sessions without user identification, which usually assume that items in a session are highly correlated with each other and center around an intense intent [26, 41].

    Another line of research models user preferences with historic item sequences spanning a longer period of time. Pioneering works adopt Markov Chains [28] and translation-based [14] methods to model the transition between items that a user interacted with sequentially. Recently, there are lots of efforts in applying different neural networks to capture users’ dynamic preferences from their sequential behaviors. GRU4Rec [17] utilizes a Gated Recurrent Neural Network (GRU) to investigate users’ sequential interactions and then GRU4Rec+ [16] is proposed as a modified version with a new class of loss function designed for the Top-K recommendation. Meanwhile, Convolutional Neural Networks (CNN) are adopted by [32, 44] to capture the sequential patterns of users’ historic interactions. While self-attention layer (transformer) [33] is proposed to be an effective replacement for RNN and CNN in handling sequential data, it is adopted in SASRec [18] to extract user preferences from the interactions in the past. However, these methods focus on modeling the sequential patterns without considering the temporal effects, leading to similar latent representations for interactions happening at different time periods or from various users.
  • This work was supported in part by NSF grant IIS-1841138
  • Sameer Agarwal, Kristin Branson, and Serge Belongie. 2006. Higher order learning with graphs. In ICML.
    Google ScholarFindings
  • James Atwood and Don Towsley. 2016. Diffusion-convolutional neural networks. In NeurIPS.
    Google ScholarFindings
  • Song Bai, Feihu Zhang, and Philip HS Torr. 2019. Hypergraph Convolution and Hypergraph Attention. arXiv preprint arXiv:1901.08150 (2019).
  • Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017).
  • Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang, and Xiaofei He. 2010. Music recommendation by unified hypergraph: combining social media information and music content. In MM.
    Google ScholarFindings
  • Zhiyong Cheng, Jialie Shen, Lei Zhu, Mohan S Kankanhalli, and Liqiang Nie. 2017. Exploiting Music Play Sequence for Music Recommendation.. In IJCAI.
    Google ScholarFindings
  • Hanjun Dai, Yichen Wang, Rakshit Trivedi, and Le Song. 2016. Recurrent coevolutionary latent feature processes for continuous-time recommendation. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.
    Google ScholarLocate open access versionFindings
  • Kaize Ding, Jundong Li, Rohit Bhanushali, and Huan Liu. 2019. Deep anomaly detection on attributed networks. In SDM.
    Google ScholarFindings
  • Kaize Ding, Yichuan Li, Jundong Li, Chenghao Liu, and Huan Liu. 201Feature Interaction-aware Graph Neural Networks. arXiv preprint arXiv:1908.07110 (2019).
  • Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph Neural Networks for Social Recommendation. In WWW.
    Google ScholarLocate open access versionFindings
  • Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. 2019. Hypergraph neural networks. In AAAI.
    Google ScholarLocate open access versionFindings
  • Mihajlo Grbovic, Vladan Radosavljevic, Nemanja Djuric, Narayan Bhamidipati, Jaikit Savla, Varun Bhagwan, and Doug Sharp. 2015. E-commerce in your inbox: Product recommendations at scale. In KDD.
    Google ScholarFindings
  • Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS.
    Google ScholarFindings
  • Ruining He, Wang-Cheng Kang, and Julian McAuley. 2017. Translation-based recommendation. In RecSys.
    Google ScholarFindings
  • Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW.
    Google ScholarLocate open access versionFindings
  • Balázs Hidasi and Alexandros Karatzoglou. 2018. Recurrent neural networks with top-k gains for session-based recommendations. In CIKM.
    Google ScholarFindings
  • Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
  • Wang-Cheng Kang and Julian McAuley. 20Self-attentive sequential recommendation. In ICDM.
    Google ScholarFindings
  • Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  • Yehuda Koren. 2009. Collaborative filtering with temporal dynamics. In KDD.
    Google ScholarFindings
  • Lei Li and Tao Li. 2013. News recommendation via hypergraph learning: encapsulation of user behavior and news content. In WSDM.
    Google ScholarFindings
  • Chen Ma, Peng Kang, and Xue Liu. 2019. Hierarchical Gating Networks for
    Google ScholarFindings
  • Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013.
    Google ScholarFindings
  • Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects. In EMNLP-IJCNLP.
    Google ScholarFindings
  • Kan Ren, Jiarui Qin, Yuchen Fang, Weinan Zhang, Lei Zheng, Weijie Bian, Guorui Zhou, Jian Xu, Yong Yu, Xiaoqiang Zhu, et al. 2019. Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction. In SIGIR.
    Google ScholarFindings
  • Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten de Rijke. 2019. RepeatNet: A repeat aware neural recommendation machine for session-based recommendation. In AAAI.
    Google ScholarFindings
  • Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In UAI.
    Google ScholarFindings
  • Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW.
    Google ScholarFindings
  • Qingquan Song, Shiyu Chang, and Xia Hu. 2019. Coupled Variational Recurrent Collaborative Filtering. In KDD.
    Google ScholarFindings
  • Weiping Song, Zhiping Xiao, Yifan Wang, Laurent Charlin, Ming Zhang, and Jian Tang. 2019. Session-based Social Recommendation via Dynamic Graph Attention Networks. In WSDM.
    Google ScholarFindings
  • Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. CIKM.
    Google ScholarLocate open access versionFindings
  • Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM.
    Google ScholarFindings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS.
    Google ScholarFindings
  • Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
  • Jianling Wang and James Caverlee. 2019. Recurrent Recommendation with Local Coherence. In WSDM.
    Google ScholarFindings
  • Jianling Wang, Kaize Ding, Ziwei Zhu, Yin Zhang, and James Caverlee. 2020. Key Opinion Leaders in Recommendation Systems: Opinion Elicitation and Diffusion. In WSDM.
    Google ScholarFindings
  • Jianling Wang, Raphael Louca, Diane Hu, Caitlin Cellier, James Caverlee, and Liangjie Hong. 2020. Time to Shop for Valentine’s Day: Shopping Occasions and Sequential Recommendation in E-commerce. In WSDM.
    Google ScholarFindings
  • Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural Graph Collaborative Filtering. In SIGIR.
    Google ScholarLocate open access versionFindings
  • Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017. Recurrent recommender networks. In WSDM.
    Google ScholarFindings
  • Le Wu, Peijie Sun, Yanjie Fu, Richang Hong, Xiting Wang, and Meng Wang. 2019. A Neural Influence Diffusion Model for Social Recommendation. arXiv preprint arXiv:1904.10322 (2019).
  • Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based recommendation with graph neural networks. In AAAI.
    Google ScholarFindings
  • Liang Xiang, Quan Yuan, Shiwan Zhao, Li Chen, Xiatian Zhang, Qing Yang, and Jimeng Sun. 2010. Temporal recommendation on graphs via long-and short-term preference fusion. In KDD.
    Google ScholarFindings
  • Dingqi Yang, Bingqing Qu, Jie Yang, and Philippe Cudre-Mauroux. 2019. Revisiting user mobility and social relationships in lbsns: a hypergraph embedding approach. In WWW.
    Google ScholarFindings
  • Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2019. A Simple Convolutional Generative Network for Next Item Recommendation. In WSDM.
    Google ScholarFindings
  • Yu Zhu, Ziyu Guan, Shulong Tan, Haifeng Liu, Deng Cai, and Xiaofei He. 2016. Heterogeneous hypergraph embedding for document recommendation. Neurocomputing 216 (2016), 150–162.
    Google ScholarLocate open access versionFindings
Your rating :