AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
CSAN: Contextual Self-Attention Network for User Sequential Recommendation.
MM '18: ACM Multimedia Conference Seoul Republic of Korea October, 2018, pp.447-455, (2018)
- With the rapid development of the Internet, some applications of sequential scenario have become pervasive and multilateral, such “Using Data Mining Technology to Analyze the Law of Population Migration”.
- Big data, Sociology, Dynamics, Demographics Machine learning AI.
- TensorFlow reinforcement learning Li Kaifu Deep Learning Timeline.
- Matrilineal system “Using Data Mining Technology to Analyze the Law of Population Migration” Data mining, Big data, Sociology of Scientific Knowledge, SSK
- With the rapid development of the Internet, some applications of sequential scenario have become pervasive and multilateral, such “Using Data Mining Technology to Analyze the Law of Population Migration”
Data mining, Big data, Sociology, Dynamics, Demographics Machine learning AI
TensorFlow reinforcement learning Li Kaifu Deep Learning Timeline
Economic sociology Demography
Matrilineal system “Using Data Mining Technology to Analyze the Law of Population Migration” Data mining, Big data, Sociology of Scientific Knowledge, SSK
- We propose a novel contextual self-attention network for the sequential recommendation, which can leverage user historical behaviors in a more effective manner and have high computational efficiency
- We propose to employ embedding network, self-attention mechanism and position encoding to deal with the heterogeneity, polysemy, and dynamic contextual dependency of user sequential behaviors
- Our work differs from the above approaches in that we introduce a unified Recurrent Neural Network (RNN)/Convolutional Neural Network (CNN)-free user behavior modeling framework based solely on self-attention for sequential recommendation whose attention mechanism works on the feature level instead of element level, and use position encoding matrices to model dynamic contextual dependency instead of time encoding
- We introduce a contextual self-attention network, Contextual Self-Attention Network (CSAN), for modeling the sequential behaviors in recommendation tasks
- CSAN is a unified framework which can model with multi-type actions and multi-modal content based solely on attention mechanism
- 4.1 Evaluation Datasets
The authors evaluate the proposed method on two real-world datasets: Amazon product dataset and Zhihu activity dataset.
- Amazon3 is an e-commerce website where users interacts with the commodity.
- We take a series of large categories including ‘Automotive’, ‘Office Products’, ‘Toys and Games’, ‘Clothing, Shoes and Jewelry’, and ‘Video Games’ for experiment.
- This set of data is notable for its high sparsity and variability.
- The authors retain some of the characteristics used to construct the user sequential behaviors
- AUC , the area under the ROC curve, is a commonly used metric for evaluating the quality of a ranking list.
- The authors report the performance of each method on the test set on both Amazon datasets and.
- Zhizhu dataset in terms of the following ranking metrics: AUC = 1 U u ∈U 1 |J ||J′|.
- Pu, j is the predicted probability that a user u ∈ U may act on i in the test set.
- A higher value of AUC indicates better performance for ranking performance.
- AUC from random guess is 0.5 and the best result is 1
- The authors introduce a contextual self-attention network, CSAN, for modeling the sequential behaviors in recommendation tasks.
- CSAN is a unified framework which can model with multi-type actions and multi-modal content based solely on attention mechanism.
- The authors analyze the proposed model on both single-type behaviors datasets (Amazon) and multitype multi-modal behaviors dataset (Zhihu).
- The experiment results show that CSAN achieves promising performances over the existing highly optimized individual models, and demonstrates its suitability for modeling complex behavior patterns
- Table1: Statistics of Amazon product dataset
- Table2: Statistics of Zhihu activity dataset
- Table3: Ranking results on Amazon datasets and Zhihu dataset (higher is better). The best performance in each case is highlighted
- Sequential Recommendation Sequential recommendation problem is usually cast as sequence prediction problem. Most existing approaches focus on Markov Chain (MC) based methods and Neural network-based methods. Scalable sequential models usually rely on MC to capture sequential patterns [7, 26], where an L-order Markov chain makes recommendations based on L previous actions. However, a major problem of MC based models is that all the components are independently combined, indicating that it makes strong independence assumptions among multiple factors . Recently, a Matrix Factorization (MF) based approach factorizes the matrix of transition probability from the current item to the next one into the latent factors . However, MF easily suffers from sparsity issues due to the power-law distributed data in the real word . Inspired by the great power of matrix factorization, Factorized Personalized Markov Chain (FPMC)  combines the power of MF and MC to factorize the transition matrix over underlying MC to model personalized sequential behaviors for the next-basket recommendation. FPMC and its variant  improve this method by factorizing this transition matrix into two latent and low-rank sub-matrices. All the MC-based methods have the same deficiency that these recommenders only obtain the local sequential behaviors between every two adjacent items.
- This work was supported in part by the National Key Research and Development Program of China (No 2017YFB1002804), the National Natural Science Foundation of China under Grants 61432019, 61720106006, 61572503 and 61702509, the Key Research Program of Frontier Sciences, CAS, Grant NO
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. CoRR abs/1409.0473 (2014).
- Chen Cheng, Haiqin Yang, Michael R Lyu, and Irwin King. 2013. Where You Like to Go Next: Successive Point-of-Interest Recommendation.. In IJCAI, Vol. 13. 2605–2611.
- Szu-Yu Chou, Yi-Hsuan Yang, Jyh-Shing Roger Jang, and Yu-Ching Lin. 2016. Addressing cold start for next-song recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 115–118.
- Junyoung Chung, ÃĞaglar GülÃğehre, Kyunghyun Cho, and Yoshua Bengio. 201Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR abs/1412.3555 (2014).
- Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. 2017. Attention-over-Attention Neural Networks for Reading Comprehension. In ACL.
- Yong Du, Wei Wang, and Liang Wang. 2015. Hierarchical recurrent neural network for skeleton based action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1110–1118.
- Shanshan Feng, Xutao Li, Yifeng Zeng, Gao Cong, Yeow Meng Chee, and Quan Yuan. 2015. Personalized Ranking Metric Embedding for Next New POI Recommendation.. In IJCAI. 2069–2075.
- Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann Dauphin. 2017. Convolutional Sequence to Sequence Learning. In ICML.
- Ruining He, Wang-Cheng Kang, and Julian McAuley. 2017. Translation-based Recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 161–169.
- Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In Data Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, 191–200.
- Liang Hu, Longbing Cao, Jian Cao, Zhiping Gu, Guandong Xu, and Dingyu Yang. 2016. Learning Informative Priors from Heterogeneous Domains to Improve Recommendation in Cold-Start User Domains. ACM Transactions on Information Systems (TOIS) 35, 2 (2016), 13.
- Minghao Hu, Yuxing Peng, and Xipeng Qiu. 2017. Reinforced mnemonic reader for machine comprehension. CoRR, abs/1705.02798 (2017).
- Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. 2014. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 1725–1732.
- Yoon Kim. 20Convolutional Neural Networks for Sentence Classification. In EMNLP.
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097–1105.
- Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International Conference on Machine Learning. 1188–1196.
- Zhouhan Lin, Minwei Feng, Cícero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 20A Structured Self-attentive Sentence Embedding. CoRR abs/1703.03130 (2017).
- Duen-Ren Liu, Chin-Hui Lai, and Wang-Jung Lee. 2009. A hybrid of sequential rules and collaborative filtering for product recommendation. Information Sciences 179, 20 (2009), 3505–3519.
- Qiang Liu, Shu Wu, and Liang Wang. 2017. Multi-behavioral sequential prediction with recurrent log-bilinear model. IEEE Transactions on Knowledge and Data Engineering 29, 6 (2017), 1254–1267.
- Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2016. Predicting the Next Location: A Recurrent Model with Spatial and Temporal Contexts.. In AAAI. 194–200.
- Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 43–52.
- Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černocky, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association.
- Tomáš Mikolov, Stefan Kombrink, Lukáš Burget, Jan Černocky, and Sanjeev Khudanpur. 2011. Extensions of recurrent neural network language model. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on. IEEE, 5528–5531.
- Tomas Mikolov, Stefan Kombrink, Anoop Deoras, Lukar Burget, and Jan Cernocky. 2011. Rnnlm-recurrent neural network language modeling toolkit. In Proc. of the 2011 ASRU Workshop. 196–201.
- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, 452–461.
- Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web. ACM, 811–820.
- Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. 2018. DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, February 2-7, 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16126
- Huan Song, Deepta Rajan, Jayaraman J. Thiagarajan, and Andreas Spanias. 2018. Attend and Diagnose: Clinical Time Series Analysis Using Attention Models. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, February 2-7, 2018. https://www.aaai.org/ocs/index.php/ AAAI/AAAI18/paper/view/16325
- Jiaxi Tang and Ke Wang. 2018. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. (2018).
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 6000–6010.
- Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, Shengxian Wan, and Xueqi Cheng. 2015. Learning hierarchical representation model for nextbasket recommendation. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. ACM, 403–412.
- Shoujin Wang, Liang Hu, Longbing Cao, Xiaoshui Huang, Defu Lian, and Wei Liu. 2018. Attention-based Transactional Context Embedding for Next-Item Recommendation. (2018).
- Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems 2, 1-3 (1987), 37–52.
- Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015.
- Shuangfei Zhai, Keng-hao Chang, Ruofei Zhang, and Zhongfei Mark Zhang. 2016. Deepintent: Learning attentions for online advertising with recurrent neural networks. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1295–1304.
- Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, and Tie-Yan Liu. 2014. Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks.. In AAAI, Vol. 14. 1369–1375.
- Lei Zheng, Vahid Noroozi, and Philip S Yu. 2017. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM, 425–434.
- Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. 2018. ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, February 2-7, 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16216
- Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018. 1059–1068. https://doi.org/10.1145/3219819.3219823