From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
CoRR(2023)
摘要
Large Language Models (LLMs) have achieved remarkable success, where
instruction tuning is the critical step in aligning LLMs with user intentions.
In this work, we investigate how the instruction tuning adjusts pre-trained
models with a focus on intrinsic changes. Specifically, we first develop
several local and global explanation methods, including a gradient-based method
for input-output attribution and techniques for interpreting patterns and
concepts in self-attention and feed-forward layers. The impact of instruction
tuning is then studied by comparing the explanations derived from the
pre-trained and instruction-tuned models. This approach provides an internal
perspective of the model shifts on a human-comprehensible level. Our findings
reveal three significant impacts of instruction tuning: 1) It empowers LLMs to
recognize the instruction parts from user prompts, and promotes the response
generation constantly conditioned on user instructions. 2) It encourages the
self-attention heads to capture more word-word relationships about instruction
verbs. 3) It encourages the feed-forward networks to rotate their pre-trained
knowledge toward user-oriented tasks. These insights contribute to a more
comprehensive understanding of instruction tuning and lay the groundwork for
future work that aims at interpreting and optimizing LLMs for various
applications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要