Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing
CoRR(2024)
摘要
Recent work using Rank-One Model Editing (ROME), a popular model editing
method, has shown that there are certain facts that the algorithm is unable to
edit without breaking the model. Such edits have previously been called
disabling edits. These disabling edits cause immediate model collapse and
limits the use of ROME for sequential editing. In this paper, we show that
disabling edits are an artifact of irregularities in the implementation of
ROME. With this paper, we provide a more stable implementation ROME, which we
call r-ROME and show that model collapse is no longer observed when making
large scale sequential edits with r-ROME, while further improving
generalization and locality of model editing compared to the original
implementation of ROME. We also provide a detailed mathematical explanation of
the reason behind disabling edits.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要