Model validation for aggregate inferences in out-of-sample prediction
arxiv(2023)
摘要
Generalization to new samples is a fundamental rationale for statistical
modeling. For this purpose, model validation is particularly important, but
recent work in survey inference has suggested that simple aggregation of
individual prediction scores does not give a good measure of the score for
population aggregate estimates. In this manuscript we explain why this occurs,
propose two scoring metrics designed specifically for this problem, and
demonstrate their use in three different ways. We show that these scoring
metrics correctly order models when compared to the true score, although they
do underestimate the magnitude of the score. We demonstrate with a problem in
survey research, where multilevel regression and poststratification (MRP) has
been used extensively to adjust convenience and low-response surveys to make
population and subpopulation estimates.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要