Toward Fact-aware Abstractive Summarization Method Using Joint Learning

Abstract Abstractive summarization obtains text semantic embedding based on the content of the source corpus to generate summaries, helping users quickly extract valid information from massive amounts of text data. The existing model suffers from poor ability to perceive the correctness of sentence formulation, resulting in the generation of summaries with distorted factual content and representations that do not match the information in the original text, i.e. factual inconsistencies. To address this issue, in this paper, the textual implication task is learned jointly with the summary task, starting from the original text and guided by template sentences. The knowledge of implicit reasoning is incorporated into the encoder model of the summary through parameter sharing to improve the model's ability to perceive facts. First, to enhance the factual consistency of the generated content, this paper uses the extractive summary algorithm Lead-3 to extract partial sentences from the original text as template sentences. Second, we construct a weakly supervised textual implication discriminator dataset on the CNN/Daily Mail. By sharing the same BERT-based text encoder, the text-implication task is jointly trained with the text-summarization task to make the encoder text-implication aware. Finally, experiments were conducted on CNN/Daily Mail to verify the accuracy and factual consistency of the generated summaries by using ROUGE metrics and BERTScore metrics. The results demonstrate that the fact-aware abstractive summarization model proposed in this paper can generate high-quality and factually consistent summaries.