Is Factuality Decoding a Free Lunch for LLMs? Evaluation on Knowledge Editing Benchmark
arxiv(2024)
摘要
The rapid development of large language models (LLMs) enables them to convey
factual knowledge in a more human-like fashion. Extensive efforts have been
made to reduce factual hallucinations by modifying LLMs with factuality
decoding. However, they also pose risks of hindering knowledge updates, as they
make models overly confident in known facts. In this work, we first revisite
the current factuality decoding methods and verified their effectiveness in
enhancing factual accuracy. Subsequently, we conduct further evaluation of
several strong factuality decoding methods on the knowledge editing benchmark.
All these decoding methods significantly diminish the performance of llama2
models compared to their original decoding, with the largest decrease being a
staggering 81.3%. This further indicates that the current existing decoding
methods still cannot perfectly address the factual hallucinations, as they
overlook the importance of preserving the flexibility for knowledge editing.
Therefore, our work suggests that research into factual alignment should
simultaneously focus on the effectiveness of knowledge editing.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要