Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
CoRR(2024)
摘要
Contemporary language models enable new opportunities for structured
reasoning with text, such as the construction and evaluation of intuitive,
proof-like textual entailment trees without relying on brittle formal logic.
However, progress in this direction has been hampered by a long-standing lack
of a clear protocol for determining what valid compositional entailment is.
This absence causes noisy datasets and limited performance gains by modern
neuro-symbolic engines. To address these problems, we formulate a consistent
and theoretically grounded approach to annotating decompositional entailment
datasets, and evaluate its impact on LLM-based textual inference. We find that
our resulting dataset, RDTE (Recognizing Decompositional Textual Entailment),
has a substantially higher internal consistency (+9
decompositional entailment datasets, suggesting that RDTE is a significant step
forward in the long-standing problem of forming a clear protocol for discerning
entailment. We also find that training an RDTE-oriented entailment classifier
via knowledge distillation and employing it in a modern neuro-symbolic
reasoning engine significantly improves results (both accuracy and proof
quality) over other entailment classifier baselines, illustrating the practical
benefit of this advance for textual inference.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要