DECIDER: A Dual-System Rule-Controllable Decoding Framework for Language Generation
arxiv(2024)
摘要
Constrained decoding approaches aim to control the meaning or style of text
generated by a Pre-trained Language Model (PLM) using specific target words
during inference. However, these methods often guide plausible continuations by
greedily selecting targets, which, while completing the task, may disrupt the
natural patterns of human language generation. In this work, we propose a novel
decoding framework, DECIDER, which enables us to program rules on how we
complete tasks to control a PLM. Differing from previous work, our framework
transforms the encouragement of target words into the encouragement of all
words that satisfy the rule. Specifically, DECIDER is a dual system where a PLM
is equipped with a First-OrderLogic (FOL) reasoner to express and evaluate the
rules, and a decision function to merge the outputs from both systems to steer
the generation. Experiments on CommonGen and PersonaChat demonstrate that
DECIDER can effectively follow given rules to achieve generation tasks in a
more human-like manner.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要