Logical Rule-Constrained Large Language Models for Document-Level Relation Extraction.

Document-level Relation Extraction (DocRE) aims to identify the relation types between entity pairs within a document. Large language models (LLMs) have garnered attention due to their strong reasoning capabilities. However, directly applying LLMs to DocRE often results in suboptimal performance, primarily due to long-context inputs inhibiting its reasoning capability. Logical rules can inject refined information into LLMs due to their explicit reasoning process. In this paper, we propose logical Rule-constrained LLMs or Ru-LLM for short, a novel framework that learns logical rules to enhance the performance of LLMs in DocRE. Specifically, Ru-LLM leverages frequent pattern mining and LLM-assisted completion to obtain the candidate predicate pool. To further refine the rule search process, Ru-LLM incorporates the Monte Carlo Tree Search algorithm to efficiently search for and generate logical rules through a four-stage process: selection, expansion, simulation, and backpropagation. Ru-LLM transforms logical rules into concise prompts to guide initial result generation, which is then subjected to a Chain-of-Thought verification process that retains only those triples satisfying all rules. Experimental results on the three datasets DWIE, DocRED, and Re-DocRED demonstrate that Ru-LLM outperforms existing implicit reasoning models, LLM-based models, and rule-based frameworks.