Training Language Models to Generate Text with Citations via Fine-grained Rewards
CoRR(2024)
摘要
While recent Large Language Models (LLMs) have proven useful in answering
user queries, they are prone to hallucination, and their responses often lack
credibility due to missing references to reliable sources. An intuitive
solution to these issues would be to include in-text citations referring to
external documents as evidence. While previous works have directly prompted
LLMs to generate in-text citations, their performances are far from
satisfactory, especially when it comes to smaller LLMs. In this work, we
propose an effective training framework using fine-grained rewards to teach
LLMs to generate highly supportive and relevant citations, while ensuring the
correctness of their responses. We also conduct a systematic analysis of
applying these fine-grained rewards to common LLM training strategies,
demonstrating its advantage over conventional practices. We conduct extensive
experiments on Question Answering (QA) datasets taken from the ALCE benchmark
and validate the model's generalizability using EXPERTQA. On LLaMA-2-7B, the
incorporation of fine-grained rewards achieves the best performance among the
baselines, even surpassing that of GPT-3.5-turbo.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要