InstructEdit: Instruction-based Knowledge Editing for Large Language Models
arxiv(2024)
摘要
Knowledge editing for large language models can offer an efficient solution
to alter a model's behavior without negatively impacting the overall
performance. However, the current approaches encounter issues with limited
generalizability across tasks, necessitating one distinct editor for each task,
significantly hindering the broader applications. To address this, we take the
first step to analyze the multi-task generalization issue in knowledge editing.
Specifically, we develop an instruction-based editing technique, termed
InstructEdit, which facilitates the editor's adaptation to various task
performances simultaneously using simple instructions. With only one unified
editor for each LLM, we empirically demonstrate that InstructEdit can improve
the editor's control, leading to an average 14.86
multi-task editing setting. Furthermore, experiments involving holdout unseen
task illustrate that InstructEdit consistently surpass previous strong
baselines. To further investigate the underlying mechanisms of
instruction-based knowledge editing, we analyze the principal components of the
editing gradient directions, which unveils that instructions can help control
optimization direction with stronger OOD generalization. Code and datasets are
available in https://github.com/zjunlp/EasyEdit.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要