Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning
CoRR(2024)
摘要
Large language models exhibit high-level commonsense reasoning abilities,
especially with enhancement methods like Chain-of-Thought (CoT). However, we
find these CoT-like methods lead to a considerable number of originally correct
answers turning wrong, which we define as the Toxic CoT problem. To interpret
and mitigate this problem, we first utilize attribution tracing and causal
tracing methods to probe the internal working mechanism of the LLM during CoT
reasoning. Through comparisons, we prove that the model exhibits information
loss from the question over the shallow attention layers when generating
rationales or answers. Based on the probing findings, we design a novel method
called RIDERS (Residual decodIng and sERial-position Swap), which compensates
for the information deficit in the model from both decoding and serial-position
perspectives. Through extensive experiments on multiple commonsense reasoning
benchmarks, we validate that this method not only significantly eliminates
Toxic CoT problems (decreased by 23.6
model's overall commonsense reasoning performance (increased by 5.5
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要