AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
Multi-gate Mixture-of-Experts to jointly optimize multi-objectives in e-commerce, and uses the Bias Deep Neural Networks to reduce the select bias in implicit feedback

Deep Multifaceted Transformers for Multi-objective Ranking in Large-Scale E-commerce Recommender Systems

CIKM '20: The 29th ACM International Conference on Information and Knowledge Management Virt..., pp.2493-2500, (2020)

被引用0|浏览351
EI
下载 PDF 全文
引用
微博一下

摘要

Recommender Systems have been playing essential roles in e-commerce portals. Existing recommendation algorithms usually learn the ranking scores of items by optimizing a single task (e.g. Click-through rate prediction) based on users' historical click sequences, but they generally pay few attention to simultaneously modeling users' multip...更多

代码

数据

0
简介
  • Recommender Systems, which aim to recommend potentially interested items for users and solve the information explosion problem, are playing critical roles in E-commerce sites (e.g., Amazon, JD.com, Alibaba) [10, 20, 41, 42], videos sharing sites (e.g., YouTube) [7], picture sharing sites (e.g., Pinterest) [36], social networks (e.g., Facebook) [11, 13] and so on.
  • In JD.com, one of the largest E-commerce sites in the world, the Recommender System serves more than 0.3 billion users in China, Thailand, Malaysia and other countries, and contributes billions of dollars for the Gross Merchandise Volume (GMV) each year.
  • Many ranking methods for recommendations have been proposed, including tree-based methods [9], deep neural networks [5, 7, 10, 40, 41], reinforcement learning [42, 43] based models and so on.
  • Designing a real-world large-scale E-commerce recommender system still faces many challenges, including:
重点内容
  • Recommender Systems, which aim to recommend potentially interested items for users and solve the information explosion problem, are playing critical roles in E-commerce sites (e.g., Amazon, JD.com, Alibaba) [10, 20, 41, 42], videos sharing sites (e.g., YouTube) [7], picture sharing sites (e.g., Pinterest) [36], social networks (e.g., Facebook) [11, 13] and so on
  • We propose a novel framework Deep Multifaceted Transformers (DMT) for the multi-objective ranking in large-scale E-commerce Recommender Systems
  • We present DMT, which exploits multiple transformers to model users’ diverse behavior sequences, utilizes Multi-gate Mixtureof-Experts to jointly optimize multi-objectives, and uses a Bias Deep Neural Network for reducing the bias in E-commerce Recommender Systems
  • We propose DMT, which exploits Deep Multifaceted
  • Multi-gate Mixture-of-Experts to jointly optimize multi-objectives in e-commerce, and uses the Bias Deep Neural Networks to reduce the select bias in implicit feedback
  • We conduct extensive experiments and demonstrate that DMT outperforms state-of-the-art baselines greatly for both click and conversion tasks
  • We conduct extensive experiments and demonstrate the effectiveness of DMT for multi-objective ranking in large-scale e-commerce Recommender Systems
方法
  • The authors introduce DMT, a Deep Multifaceted Transformers based framework for multi-objective ranking in E-commerce Recommender Systems.
  • The inputs of DMT can be divided into two parts: categorical features and dense features.
  • The most useful categorical features in e-commerce recommender systems are users’ diverse behaviors.
  • As Deep Neural Networks are sensitive to the scaling of their inputs, the authors use the Z-score Normalization method to normalize the dense features
结果
  • 6.1 Comparison with Baselines

    Table 1 shows the experimental results of different methods for the click prediction and order prediction tasks.
  • To investigate whether the existing dense features for the previous GBDT model can help improve the model’s performance, the authors further incorporate them in the models, and demonstrate the results in the lower part of the table
  • From this table, the authors can find that: (1) DIN performs better than DNN model by utilizing the attention mechanism, and DIEN can achieve better performance than DIN by further modeling the sequential information in user’s historical sequence.
  • The reason is that the the dense features, which have been designed and improved for more than five years, already contain about 200 dense features to model the information in users’ behavior sequences
结论
  • The authors propose DMT, which exploits Deep Multifaceted

    Transformers to model users’ diverse behavior sequences, utilizes

    Multi-gate Mixture-of-Experts to jointly optimize multi-objectives in e-commerce, and uses the Bias Deep Neural Networks to reduce the select bias in implicit feedback.
  • The authors propose DMT, which exploits Deep Multifaceted.
  • Transformers to model users’ diverse behavior sequences, utilizes.
  • Multi-gate Mixture-of-Experts to jointly optimize multi-objectives in e-commerce, and uses the Bias Deep Neural Networks to reduce the select bias in implicit feedback.
  • The authors conduct extensive experiments and demonstrate the effectiveness of DMT for multi-objective ranking in large-scale e-commerce Recommender Systems.
  • A/B testing in JD Recommeder Systems further demonstrates that.
  • DMT can achieve substantial improvements in business
总结
  • Introduction:

    Recommender Systems, which aim to recommend potentially interested items for users and solve the information explosion problem, are playing critical roles in E-commerce sites (e.g., Amazon, JD.com, Alibaba) [10, 20, 41, 42], videos sharing sites (e.g., YouTube) [7], picture sharing sites (e.g., Pinterest) [36], social networks (e.g., Facebook) [11, 13] and so on.
  • In JD.com, one of the largest E-commerce sites in the world, the Recommender System serves more than 0.3 billion users in China, Thailand, Malaysia and other countries, and contributes billions of dollars for the Gross Merchandise Volume (GMV) each year.
  • Many ranking methods for recommendations have been proposed, including tree-based methods [9], deep neural networks [5, 7, 10, 40, 41], reinforcement learning [42, 43] based models and so on.
  • Designing a real-world large-scale E-commerce recommender system still faces many challenges, including:
  • Methods:

    The authors introduce DMT, a Deep Multifaceted Transformers based framework for multi-objective ranking in E-commerce Recommender Systems.
  • The inputs of DMT can be divided into two parts: categorical features and dense features.
  • The most useful categorical features in e-commerce recommender systems are users’ diverse behaviors.
  • As Deep Neural Networks are sensitive to the scaling of their inputs, the authors use the Z-score Normalization method to normalize the dense features
  • Results:

    6.1 Comparison with Baselines

    Table 1 shows the experimental results of different methods for the click prediction and order prediction tasks.
  • To investigate whether the existing dense features for the previous GBDT model can help improve the model’s performance, the authors further incorporate them in the models, and demonstrate the results in the lower part of the table
  • From this table, the authors can find that: (1) DIN performs better than DNN model by utilizing the attention mechanism, and DIEN can achieve better performance than DIN by further modeling the sequential information in user’s historical sequence.
  • The reason is that the the dense features, which have been designed and improved for more than five years, already contain about 200 dense features to model the information in users’ behavior sequences
  • Conclusion:

    The authors propose DMT, which exploits Deep Multifaceted

    Transformers to model users’ diverse behavior sequences, utilizes

    Multi-gate Mixture-of-Experts to jointly optimize multi-objectives in e-commerce, and uses the Bias Deep Neural Networks to reduce the select bias in implicit feedback.
  • The authors propose DMT, which exploits Deep Multifaceted.
  • Transformers to model users’ diverse behavior sequences, utilizes.
  • Multi-gate Mixture-of-Experts to jointly optimize multi-objectives in e-commerce, and uses the Bias Deep Neural Networks to reduce the select bias in implicit feedback.
  • The authors conduct extensive experiments and demonstrate the effectiveness of DMT for multi-objective ranking in large-scale e-commerce Recommender Systems.
  • A/B testing in JD Recommeder Systems further demonstrates that.
  • DMT can achieve substantial improvements in business
表格
  • Table1: Performance of different methods for Click and Order prediction. “∗” indicates the statistically significant improvements (i.e., p-value < 0.01) over the best baseline
  • Table2: Statistics of the Datset
  • Table3: Performance of position embedding in DMT
  • Table4: Performance of multi-task learning in DMT
  • Table5: Performance of Bias Deep Neural Network in DMT
  • Table6: Performance in online A/B Testing in JD Recommender Systems. “∗” indicates the statistically significant improvements (i.e., p-value < 0.01) over the baseline
Download tables as Excel
相关工作
  • 2.1 CTR prediction

    Click-through rate (CTR) prediction, which aims to estimate the probability of a user clicking on the item, is one of the long-standing core tasks in industrial applications, such as Recommender Systems [5, 7, 37], Search Engine [35], Advertising [13, 40, 41] and so on. GBDT [9] is one of the most popular and successful methods for CTR prediction in industrial systems [13, 35]. It has the advantage of simplicity, effectiveness, good explainability, flexible extensibility and so on. In recent years, deep learning based methods have achieved appealing performance for CTR prediction. These methods [5, 7] follow the Embedding&MLP paradigm: large-scale sparse input features are firstly mapped into low dimensional embedding vectors, and then concatenated together to fed into the multilayer perceptron (MLP) to learn the nonlinear relations among features. Wide&Deep [5] combines wide linear models and deep neural networks for recommender systems. Some work [2, 12, 28, 29] focus on the feature interaction problem. They can be regarded as complement work with our approach. State-of-the-art methods have found the effectiveness of modeling users’ historical behaviors for CTR prediction [8, 10, 16, 17, 26, 27, 31, 40, 41]. DIN [41] notices that a user may have multiple interests and uses attention mechanism to learn the representation of user interests from historical behaviors with respect to a certain candidate item. It achieves better performance than the simple Embedding&MLP method [7]. To further consider the sequential information in users’ click sequence, DIEN [40] uses two layers of GRU to model the click sequence and capture the evolution of the user’s interest. HUP [10] exploits Pyramid Recurrent Neural Networks to model users’ hierarchical interests in categories and items. Recently, some works [3, 8, 18, 39] attempt to exploit Self-Attention Neural Networks to model user’s behavior sequence. However, existing methods usually focus on modeling a single type of user’s behavior sequence or consider a single objective. How to effectively model users’ multiple types of behaviors sequences on items for multiple objectives in industrial Recommender Systems still remains as an open problem.
基金
  • • We conduct extensive experiments and demonstrate that DMT outperforms state-of-the-art baselines greatly for both click and conversion tasks
研究对象与分析
separate Deep Interest Transformers: 3
4.2.1 Deep Multifaceted Transformers. To capture each user’s multifaceted interest, we use three separate Deep Interest Transformers (they have different parameters) to model the user’s click sequence, cart sequence and order sequence, and learn the user’s short-term, middle-term and long-term interest vectors respectively. Multiple Objectives Task 1

引用论文
  • Aman Agarwal, Kenta Takatsu, Ivan Zaitsev, and Thorsten Joachims. 2019. A General Framework for Counterfactual Learning-to-Rank. In SIGIR’19. 5–14.
    Google ScholarLocate open access versionFindings
  • Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, and Ed H Chi. 2018. Latent cross: Making use of context in recurrent recommender systems. In WSDM’18. 46–54.
    Google ScholarLocate open access versionFindings
  • Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou. 2019. Behavior sequence transformer for e-commerce recommendation in Alibaba. In DLP-KDD’19. 1–4.
    Google ScholarLocate open access versionFindings
  • Weijian Chen, Yulong Gu, Zhaochun Ren, Xiangnan He, Hongtao Xie, Tong Guo, Dawei Yin, and Yongdong Zhang. 2019. Semi-supervised user profiling with heterogeneous graph attention networks. In IJCAI’19. 2116–2122.
    Google ScholarFindings
  • Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016.
    Google ScholarLocate open access versionFindings
  • Aleksandr Chuklin, Pavel Serdyukov, and Maarten De Rijke. 2013. Click modelbased information retrieval metrics. In SIGIR’13. 493–502.
    Google ScholarLocate open access versionFindings
  • Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In RecSys’16. 191–198.
    Google ScholarLocate open access versionFindings
  • Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, and Keping Yang. 2019. Deep session interest network for click-through rate prediction. In IJCAI’19. 2301–2307.
    Google ScholarFindings
  • Jerome H Friedman. 2001. Greedy function approximation: a gradient boosting machine. Annals of statistics (2001), 1189–1232.
    Google ScholarLocate open access versionFindings
  • Yulong Gu, Zhuoye Ding, Shuaiqiang Wang, and Dawei Yin. 2020. Hierarchical User Profiling for E-commerce Recommender Systems. In WSDM’20. 223–231.
    Google ScholarLocate open access versionFindings
  • Yulong Gu, Jiaxing Song, Weidong Liu, and Lixin Zou. 2016. HLGPS: a home location global positioning system in location-based social networks. In ICDM’16. 901–906.
    Google ScholarLocate open access versionFindings
  • Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In IJCAI’17. 1725–1731.
    Google ScholarLocate open access versionFindings
  • Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. 2014. Practical lessons from predicting clicks on ads at facebook. In ADKDD’14. 1–9.
    Google ScholarLocate open access versionFindings
  • Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased learning-to-rank with biased feedback. In WSDM’17. 781–789.
    Google ScholarLocate open access versionFindings
  • Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recommendation. In ICDM’18. 197–206.
    Google ScholarLocate open access versionFindings
  • Chenyi Lei, Shouling Ji, and Zhao Li. 2019. TiSSA: A Time Slice Self-Attention Approach for Modeling Sequential User Behaviors. In WWW’19. 2964–2970.
    Google ScholarFindings
  • Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Huan Zhao, Pipei Huang, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. 2019. Multi-interest network with dynamic routing for recommendation at Tmall. In CIKM’19.
    Google ScholarFindings
  • Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time Interval Aware SelfAttention for Sequential Recommendation. In WSDM’20. 322–330.
    Google ScholarLocate open access versionFindings
  • Xiao Lin, Hongjie Chen, Changhua Pei, Fei Sun, Xuanji Xiao, Hanxiao Sun, Yongfeng Zhang, Wenwu Ou, and Peng Jiang. 20A pareto-efficient algorithm for multiple objective optimization in e-commerce recommendation. In RecSys’19. 20–28.
    Google ScholarLocate open access versionFindings
  • Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing 7, 1 (2003), 76–80.
    Google ScholarLocate open access versionFindings
  • Yiding Liu, Yulong Gu, Zhuoye Ding, Junchao Gao, Ziyi Guo, Yongjun Bao, and Weipeng Yan. 2020. Decoupled Graph Convolution Network for Inferring Substitutable and Complementary Items. In CIKM’20.
    Google ScholarFindings
  • Yichao Lu, Ruihai Dong, and Barry Smyth. 2018. Why I like it: multi-task learning for recommendation and explanation. In RecSys’18. 4–12.
    Google ScholarLocate open access versionFindings
  • Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-ofexperts. In KDD’18. 1930–1939.
    Google ScholarLocate open access versionFindings
  • Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In SIGIR’18. 1137–1140.
    Google ScholarLocate open access versionFindings
  • Julian McAuley, Rahul Pandey, and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products. In KDD’15. 785–794.
    Google ScholarLocate open access versionFindings
  • Wentao Ouyang, Xiuwu Zhang, Li Li, Heng Zou, Xin Xing, Zhaojie Liu, and Yanlong Du. 2019. Deep spatio-temporal neural networks for click-through rate prediction. In KDD’19. 2078–2086.
    Google ScholarFindings
  • Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In KDD’19. 2671–2679.
    Google ScholarFindings
  • Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In KDD’16. 255–262.
    Google ScholarLocate open access versionFindings
  • Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. Autoint: Automatic feature interaction learning via selfattentive neural networks. In CIKM’19. 1161–1170.
    Google ScholarFindings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS’17. 5998–6008.
    Google ScholarLocate open access versionFindings
  • Huizhao Wang, Guanfeng Liu, Yan Zhao, Bolong Zheng, Pengpeng Zhao, and Kai Zheng. 2019. DMFP: A Dynamic Multi-faceted Fine-Grained Preference Model for Recommendation. In ICDM’19. 608–617.
    Google ScholarLocate open access versionFindings
  • Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale commodity embedding for e-commerce recommendation in alibaba. In KDD’18. 839–848.
    Google ScholarLocate open access versionFindings
  • Shanfeng Wang, Maoguo Gong, Haoliang Li, and Junwei Yang. 2016. Multiobjective optimization for long tail recommendation. Knowledge-Based Systems (2016), 145–155.
    Google ScholarLocate open access versionFindings
  • Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork. 2016. Learning to rank with selection bias in personal search. In SIGIR’16. 115–124.
    Google ScholarLocate open access versionFindings
  • Dawei Yin, Yuening Hu, Jiliang Tang, Tim Daly, Mianwei Zhou, Hua Ouyang, Jianhui Chen, Changsung Kang, Hongbo Deng, Chikashi Nobata, et al. 2016. Ranking relevance in yahoo search. In KDD’16. 323–332.
    Google ScholarLocate open access versionFindings
  • Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In KDD’18. 974–983.
    Google ScholarLocate open access versionFindings
  • Jiaxuan You, Yichen Wang, Aditya Pal, Pong Eksombatchai, Chuck Rosenburg, and Jure Leskovec. 2019. Hierarchical temporal convolutional networks for dynamic recommender systems. In WWW’19. 2236–2246.
    Google ScholarFindings
  • Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending what video to watch next: a multitask ranking system. In RecSys’19. 43–51.
    Google ScholarLocate open access versionFindings
  • Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. 2018. Atrank: An attention-based user behavior modeling framework for recommendation. In AAAI’18.
    Google ScholarFindings
  • Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In AAAI’19. 5941–5948.
    Google ScholarFindings
  • Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD’18. 1059–1068.
    Google ScholarLocate open access versionFindings
  • Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, and Dawei Yin. 2019. Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems. In KDD’19. 2810–2818.
    Google ScholarLocate open access versionFindings
  • Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Huang, and Dawei Yin. 2020. Neural Interactive Collaborative Filtering. In SIGIR’20. 749–758.
    Google ScholarLocate open access versionFindings
作者
Yulong Gu
Yulong Gu
Shuaiqiang Wang
Shuaiqiang Wang
Yiding Liu
Yiding Liu
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科