Mastering Symbolic Operations: Augmenting Language Models with Compiled Neural Networks
arXiv (Cornell University)(2023)
摘要
Language models' (LMs) proficiency in handling deterministic symbolic
reasoning and rule-based tasks remains limited due to their dependency implicit
learning on textual data. To endow LMs with genuine rule comprehension
abilities, we propose "Neural Comprehension" - a framework that synergistically
integrates compiled neural networks (CoNNs) into the standard transformer
architecture. CoNNs are neural modules designed to explicitly encode rules
through artificially generated attention weights. By incorporating CoNN
modules, the Neural Comprehension framework enables LMs to accurately and
robustly execute rule-intensive symbolic tasks. Extensive experiments
demonstrate the superiority of our approach over existing techniques in terms
of length generalization, efficiency, and interpretability for symbolic
operations. Furthermore, it can be applied to LMs across different model
scales, outperforming tool-calling methods in arithmetic reasoning tasks while
maintaining superior inference efficiency. Our work highlights the potential of
seamlessly unifying explicit rule learning via CoNNs and implicit pattern
learning in LMs, paving the way for true symbolic comprehension capabilities.
更多查看译文
关键词
language models,neural networks,operations
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要