Metric-aware LLM inference for regression and scoring
arxiv(2024)
摘要
Large language models (LLMs) have demonstrated strong results on a range of
NLP tasks. Typically, outputs are obtained via autoregressive sampling from the
LLM's underlying distribution. Building on prior work on Minimum Bayes Risk
Decoding, we show that this inference strategy can be suboptimal for a range of
regression and scoring tasks, and associated evaluation metrics. As a remedy,
we propose metric aware LLM inference: a decision theoretic approach optimizing
for custom regression and scoring metrics at inference time. We report
improvements over baselines on academic benchmarks and publicly available
models.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要