Navigating the OverKill in Large Language Models
CoRR(2024)
摘要
Large language models are meticulously aligned to be both helpful and
harmless. However, recent research points to a potential overkill which means
models may refuse to answer benign queries. In this paper, we investigate the
factors for overkill by exploring how models handle and determine the safety of
queries. Our findings reveal the presence of shortcuts within models, leading
to an over-attention of harmful words like 'kill' and prompts emphasizing
safety will exacerbate overkill. Based on these insights, we introduce
Self-Contrastive Decoding (Self-CD), a training-free and model-agnostic
strategy, to alleviate this phenomenon. We first extract such over-attention by
amplifying the difference in the model's output distributions when responding
to system prompts that either include or omit an emphasis on safety. Then we
determine the final next-token predictions by downplaying the over-attention
from the model via contrastive decoding. Empirical results indicate that our
method has achieved an average reduction of the refusal rate by 20% while
having almost no impact on safety.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要