Whole Page Optimization with Global Constraints

pp. 3153-3161, 2019.

Cited by: 0|Views20
EI
Weibo:
We study the problem of whole page optimization with global constraints of the type that require a certain number of impressions for certain widgets across all home page impressions

Abstract:

The Amazon video homepage is the primary gateway for customers looking to explore the large collection of content, and finding something interesting to watch. Typically, the page is personalized for a customer, and consists of a series of widgets or carousels, with each widget containing multiple items (e.g., movies, TV shows etc). Rankin...More

Code:

Data:

0
Introduction
  • Video services like Netflix, Amazon Video, or Youtube offer a vast and diverse selection of digital content for consumption.
  • Users typically start from the home page to explore and find something interesting to watch.
  • Different users have different tastes, the home page is usually personalized.
  • One effective strategy for personalization is to group content logically, and present them as carousels or widgets to the user.
  • Grouping content into widgets and providing interpretable labels for them is an interesting problem with existing literature [23], but is not the focus of this paper.
  • The authors assume that a large selection of widgets is available for display on the homepage
Highlights
  • Video services like Netflix, Amazon Video, or Youtube offer a vast and diverse selection of digital content for consumption
  • We study the problem of whole page optimization with global constraints of the type that require a certain number of impressions for certain widgets across all home page impressions
  • We propose a holistic framework for constrained whole page optimization to tackle both whole-page diversity and global constraints
  • Our model achieved 25% higher page reward and satisfied impression constrained compared to slotting approach
  • We present a primal-dual framework that encompasses relevance, diversity and global constraints for ranking widgets
  • We have empirical shown that our framework increases diversity of widgets and satisfy constraints but at the same time improve relevance
Results
  • Evaluation Metrics

    The authors measure the prediction accuracy and the homepage quality of whole page models using standard AUC and Precision@K.
  • Precision@K is defined as the number of positive widgets in top-K slot of each homepage (K ≤ n) and is normalized by K.
  • The authors measure the diversity degree of composed homepages using the standard pair-wise similarity (Div-pair@n) metrics [7].
  • Recall that I (Cn ) ∈ Rm counts the number of widgets from each of the m-categories on each page Cn and let I (Cn ) be its l1 normalized version.
  • Div-pair@n measures the average pair-wise dis-similarity between all widgets on the page, higher value indicates more diverse homepages.
  • To quantify the violation of the imposed constraints, i.e., the derivation from targeted product category balance bt , the authors calculate the percentage miss of all the constraints as miss@n
Conclusion
  • The authors present a primal-dual framework that encompasses relevance, diversity and global constraints for ranking widgets.
  • The authors have empirical shown that the framework increases diversity of widgets and satisfy constraints but at the same time improve relevance.
  • Extensive online A/B testing showed that the framework can improve customer engagement without affecting any product categories
Summary
  • Introduction:

    Video services like Netflix, Amazon Video, or Youtube offer a vast and diverse selection of digital content for consumption.
  • Users typically start from the home page to explore and find something interesting to watch.
  • Different users have different tastes, the home page is usually personalized.
  • One effective strategy for personalization is to group content logically, and present them as carousels or widgets to the user.
  • Grouping content into widgets and providing interpretable labels for them is an interesting problem with existing literature [23], but is not the focus of this paper.
  • The authors assume that a large selection of widgets is available for display on the homepage
  • Objectives:

    The authors' goal is to optimize and personalize the vertical composition of all widget. With these notations in place, the goal is to jointly optimize the |Ut | pages, argmax.
  • Results:

    Evaluation Metrics

    The authors measure the prediction accuracy and the homepage quality of whole page models using standard AUC and Precision@K.
  • Precision@K is defined as the number of positive widgets in top-K slot of each homepage (K ≤ n) and is normalized by K.
  • The authors measure the diversity degree of composed homepages using the standard pair-wise similarity (Div-pair@n) metrics [7].
  • Recall that I (Cn ) ∈ Rm counts the number of widgets from each of the m-categories on each page Cn and let I (Cn ) be its l1 normalized version.
  • Div-pair@n measures the average pair-wise dis-similarity between all widgets on the page, higher value indicates more diverse homepages.
  • To quantify the violation of the imposed constraints, i.e., the derivation from targeted product category balance bt , the authors calculate the percentage miss of all the constraints as miss@n
  • Conclusion:

    The authors present a primal-dual framework that encompasses relevance, diversity and global constraints for ranking widgets.
  • The authors have empirical shown that the framework increases diversity of widgets and satisfy constraints but at the same time improve relevance.
  • Extensive online A/B testing showed that the framework can improve customer engagement without affecting any product categories
Tables
  • Table1: Performance metric of our model without any global constraints (WPO-free) against various baselines. (B) indicates the baseline for percentage calculation. All numbers are reported in percentage lift w.r.t. baseline(s)
  • Table2: Performance metric of global constraints optimization (WPO-Global) against pure model-based diversity (WPO-Free) and slotting/pinning approach (Slot). All metrics are reported in percentage lift w.r.t baseline(s) indicated as (B)
  • Table3: Coverage of a product category X by the customer streaming propensity in product category X
Download tables as Excel
Related work
  • We connect two different threads of prior art to our problem statement in Eq (1a, 1b) and show how they can be viewed as special cases of Eq (1a, 1b). In various applications such as widget selection [1, 5, 24], sponsored advertisement [8, 19, 21], information retrieval [13, 14], and multi-arm bandits [3, 22, 25, 29], global constraints have been used to model business targets, monetary budgets, risk capacities, etc. Most of the prior art optimize the modular objective function f (·) which can be viewed as assuming n = 1 or no widgets interactions in f (·) in Eq (1a). These formulation models relevance through linear function and incorporates either local or global constraints. These prior art fails to capture the whole-page interactions or diversity of contents.

    Another line of research on whole-page optimization attempts to design page objective function f (·|u, w) to model whole-page widgets interactions. One widely adopted approach is to incorporate widget or item diversity into f (·) [4, 7, 20, 26, 28]. These approaches optimize a submodular function that measures the diversity degree of a collection of items. Recent work in [18, 27] explicitly included pair-wise widget interactions as features in f (·). These approaches formulate the problem as multivariate optimization to compose the page holistically. All these studies, however, ignore constraints g(·) in Eq (1b). In this case, the problem in Eq (1a) reduces to optimizing |Ut | number of pages independently.
Reference
  • Deepak Agarwal, Shaunak Chatterjee, Yang Yang, and Liang Zhang. 2015. Constrained optimization for homepage relevance. In Proceedings of the 24th International Conference on World Wide Web. 375–384.
    Google ScholarLocate open access versionFindings
  • Deepak Agarwal, Bee-Chung Chen, and Pradheep Elango. 2009. Spatiotemporal models for estimating click-through rate. In Proceedings of the 18th international conference on World wide web. ACM, 21–30.
    Google ScholarLocate open access versionFindings
  • Shipra Agrawal and Nikhil Devanur. 2016. Linear contextual bandits with knapsacks. In Advances in Neural Information Processing Systems. 3450–3458.
    Google ScholarLocate open access versionFindings
  • Amr Ahmed, Choon Hui Teo, S.V.N. Vishwanathan, and Alex Smola. 2012. Fair and Balanced: Learning to Present News Stories. In Proceedings of the Fifth ACM International Conference on Web Search and Data Mining. 333–342.
    Google ScholarLocate open access versionFindings
  • K. Basu, S. Chatterjee, and A. Saha. 2016. Constrained MultiSlot Optimization for Ranking Recommendations. arXiv preprint arXiv:1602.04391 (2016).
    Findings
  • Kinjal Basu, Ankan Saha, and Shaunak Chatterjee. 2017. Large-Scale Quadratically Constrained Quadratic Program via Low-Discrepancy Sequences. In Advances in Neural Information Processing Systems. 2297– 2307.
    Google ScholarLocate open access versionFindings
  • Laming Chen, Guoxin Zhang, and Hanning Zhou. 2018. Improving the Diversity of Top-N Recommendation via Determinantal Point Process. In Advances in Neural Information Processing Systems.
    Google ScholarLocate open access versionFindings
  • Ye Chen, Weiguo Liu, Jeonghee Yi, Anton Schwaighofer, and Tak W Yan. 2013. Query clustering based on bid landscape for sponsored search auction optimization. In Proc.of the 19th ACM SIGKDD Int. Conf. on Knowledge discovery and data mining. ACM, 1150–1158.
    Google ScholarLocate open access versionFindings
  • K. Christakopoulou, J. Kawale, and A. Banerjee. 2017. Recommendation with Capacity Constraints. In Proceedings of the 2017 ACM International Conference on Information and Knowledge Management. 1439–1448.
    Google ScholarLocate open access versionFindings
  • C. Clarke, M. Kolla, G. V Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon. 2008. Novelty and diversity in information retrieval evaluation. In Proceedings of the 31st ACM SIGIR conference on Research and development in information retrieval. ACM, 659–666.
    Google ScholarLocate open access versionFindings
  • Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 191–198.
    Google ScholarLocate open access versionFindings
  • A. Das, M. Datar, A. Garg, and S. Rajaram. 2007. Google news personalization: scalable online collaborative filtering. In Proceedings of the 16th international conference on World Wide Web. ACM, 271–280.
    Google ScholarLocate open access versionFindings
  • Nikhil R Devanur, Jugal Garg, Ruta Mehta, Vijay V Vaziranb, and Sadra Yazdanbod. 2018. A New Class of Combinatorial Markets with Covering Constraints: Algorithms and Applications. In Proceedings of the TwentyNinth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, 2311–2325.
    Google ScholarLocate open access versionFindings
  • Nikhil R Devanur, Zhiyi Huang, Nitish Korula, Vahab S Mirrokni, and Qiqi Yan. 2016. Whole-page optimization and submodular welfare maximization with online bidders. ACM Transactions on Economics and Computation (TEAC) 4, 3 (2016), 14.
    Google ScholarLocate open access versionFindings
  • Thore Graepel, Joaquin Quinonero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine. In Proceedings of International Conference on Machine Learning 2011.
    Google ScholarLocate open access versionFindings
  • R. Gupta, G. Liang, H. Tseng, Ravi K. Holur V., X. Chen, and R. Rosales. 20Email volume optimization at LinkedIn. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 97–106.
    Google ScholarLocate open access versionFindings
  • X. He, J. Pan, Ou Jin, T. Xu, B. Liu, T. Xu, Y. Shi, A. Atallah, R. Herbrich, and S. Bowers. 2014. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising. ACM, 1–9.
    Google ScholarLocate open access versionFindings
  • Daniel N Hill, Houssam Nassif, Yi Liu, Anand Iyer, and SVN Vishwanathan. 2017. An Efficient Bandit Algorithm for Realtime Multivariate Optimization. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1813–1821.
    Google ScholarLocate open access versionFindings
  • Jim C Huang, Rodolphe Jenatton, and Cédric Archambeau. 2016. Online Dual Decomposition for Performance and Delivery-Based Distributed Ad Allocation. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 117–126.
    Google ScholarLocate open access versionFindings
  • Neil Hurley and Mi Zhang. 2011. Novelty and diversity in top-n recommendation–analysis and evaluation. ACM Transactions on Internet Technology (TOIT) 10, 4 (2011), 14.
    Google ScholarLocate open access versionFindings
  • Rodolphe Jenatton, Jim Huang, and Cedric Archambeau. 2016. Adaptive Algorithms for Online Convex Optimization with Long-term Constraints. In International Conference on Machine Learning. 402–411.
    Google ScholarLocate open access versionFindings
  • Mehrdad Mahdavi, Rong Jin, and Tianbao Yang. 2012. Trading regret for efficiency: online convex optimization with long term constraints. Journal of Machine Learning Research 13, Sep (2012), 2503–2528.
    Google ScholarLocate open access versionFindings
  • Georgina Peake and Jun Wang. 2018. Explanation Mining: Post Hoc Interpretability of Latent Factor Models for Recommendation Systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’18). 2060–2069.
    Google ScholarLocate open access versionFindings
  • Parikshit Shah, Akshay Soni, and Troy Chevalier. 2017. Online Ranking with Constraints: A Primal-Dual Algorithm and Applications to Web Traffic-Shaping. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 405–414.
    Google ScholarLocate open access versionFindings
  • Wen Sun, Debadeepta Dey, and Ashish Kapoor. 2017. Safety-Aware Algorithms for Adversarial Contextual Bandit. In International Conference on Machine Learning. 3280–3288.
    Google ScholarLocate open access versionFindings
  • Choon Hui Teo, Houssam Nassif, Daniel Hill, Sriram Srinivasan, Mitchell Goodman, Vijai Mohan, and SVN Vishwanathan. 2016. Adaptive, personalized diversity for visual discovery. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 35–38.
    Google ScholarLocate open access versionFindings
  • Yue Wang, Dawei Yin, Luo Jie, Pengyuan Wang, Makoto Yamada, Yi Chang, and Qiaozhu Mei. 2016. Beyond ranking: Optimizing wholepage presentation. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining. ACM, 103–112.
    Google ScholarLocate open access versionFindings
  • M. Wilhelm, A. Ramanathan, A. Bonomo, S. Jain, E. Chi, and J. Gillenwater. 2018. Practical Diversified Recommendations on YouTube with Determinantal Point Processes. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2165–2173.
    Google ScholarLocate open access versionFindings
  • Hao Yu, Michael Neely, and Xiaohan Wei. 2017. Online Convex Optimization with Stochastic Constraints. In Advances in Neural Information Processing Systems. 1427–1437.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments