KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants
CoRR(2024)
摘要
An effective multi-turn instruction-following assistant can be developed by
creating a simulator that can generate useful interaction data. Apart from
relying on its intrinsic weights, an ideal user simulator should also be able
to bootstrap external knowledge rapidly in its raw form to simulate the
multifarious diversity of text available over the internet. Previous user
simulators generally lacked diversity, were mostly closed domain, and
necessitated rigid schema making them inefficient to rapidly scale to
incorporate external knowledge. In this regard, we introduce, Kaucus, a
Knowledge-Augmented User Simulator framework, to outline a process of creating
diverse user simulators, that can seamlessly exploit external knowledge as well
as benefit downstream assistant model training. Through two GPT-J based
simulators viz., a Retrieval Augmented Simulator and a Summary Controlled
Simulator we generate diverse simulator-assistant interactions. Through reward
and preference model-based evaluations, we find that these interactions serve
as useful training data and create more helpful downstream assistants. We also
find that incorporating knowledge through retrieval augmentation or summary
control helps create better assistants.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要