Uncertainty Calibration for Counterfactual Propensity Estimation in Recommendation
arxiv(2023)
摘要
Post-click conversion rate (CVR) is a reliable indicator of online customers'
preferences, making it crucial for developing recommender systems. A major
challenge in predicting CVR is severe selection bias, arising from users'
inherent self-selection behavior and the system's item selection process. To
mitigate this issue, the inverse propensity score (IPS) is employed to weight
the prediction error of each observed instance. However, current propensity
score estimations are unreliable due to the lack of a quality measure. To
address this, we evaluate the quality of propensity scores from the perspective
of uncertainty calibration, proposing the use of expected calibration error
(ECE) as a measure of propensity-score quality. We argue that the performance
of IPS-based recommendations is hampered by miscalibration in propensity
estimation. We introduce a model-agnostic calibration framework for
propensity-based debiasing of CVR predictions. Theoretical analysis on bias and
generalization bounds demonstrates the superiority of calibrated propensity
estimates over uncalibrated ones. Experiments conducted on the Coat, Yahoo and
KuaiRand datasets show improved uncertainty calibration, as evidenced by lower
ECE values, leading to enhanced CVR prediction outcomes.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要