Implicit meta-learning may lead language models to trust more reliable sources
arxiv(2023)
摘要
We demonstrate that LLMs may learn indicators of document usefulness and
modulate their updates accordingly. We introduce random strings ("tags") as
indicators of usefulness in a synthetic fine-tuning dataset. Fine-tuning on
this dataset leads to implicit meta-learning (IML): in further fine-tuning, the
model updates to make more use of text that is tagged as useful. We perform a
thorough empirical investigation of this phenomenon, finding (among other
things) that (i) it occurs in both pretrained LLMs and those trained from
scratch, as well as on a vision task, and (ii) larger models and smaller batch
sizes tend to give more IML. We also use probing to examine how IML changes the
way models store knowledge in their parameters. Finally, we reflect on what our
results might imply about capabilities, risks, and controllability of future AI
systems. Our code can be found at
https://github.com/krasheninnikov/internalization.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要