Composing Elementary Discourse Units in Abstractive Summarization

Zhenwen Li
Zhenwen Li
Wenhao Wu
Wenhao Wu

ACL, pp. 6191-6196, 2020.

Cited by: 0|Views46
EI
Weibo:
We argue that elementary discourse unit is a more appropriate textual unit of content selection than the sentence unit in abstractive summarization

Abstract:

In this paper, we argue that elementary discourse unit (EDU) is a more appropriate textual unit of content selection than the sentence unit in abstractive summarization. To well handle the problem of composing EDUs into an informative and fluent summary, we propose a novel summarization method that first designs an EDU selection model to ...More
0
Full Text
Bibtex
Weibo
Introduction
  • Abstractive summarization focuses on generating fluent and concise text from the original input document and has achieved considerable performance improvement with the rapid development of deep learning technology (See et al, 2017; Paulus et al, 2017; Celikyilmaz et al, 2018; Gehrmann et al, 2018).
  • The recently popular and practical paradigm usually generates summary sentences by independently compressing or rewriting each pre-extracted sentence, which is from the source documents (Chen and Bansal, 2018; Lebanoff et al, 2019).
  • A single document sentence usually cannot provide enough information that a summary sentence expresses, which is supported by the recent study of Lebanoff et al (2019).
  • The authors hope to seek a new summary composition unit which is more information-intensive and elementary than sentence
Highlights
  • Abstractive summarization focuses on generating fluent and concise text from the original input document and has achieved considerable performance improvement with the rapid development of deep learning technology (See et al, 2017; Paulus et al, 2017; Celikyilmaz et al, 2018; Gehrmann et al, 2018)
  • Compared to Fast-Abs which is similar to EDUSum in model architecture, EDUSum achieves better performance with respect to the three metrics, showing elementary discourse unit is more informative than sentence and appropriate to be the basic selection unit in summarization
  • We design a model EDUSumsel+RL which is similar to EDUSum except that it does not include the elementary discourse unit fusion module and directly concatenates the selected elementary discourse unit as a summary
  • EDUSumsel+RL performs worse with respect to R-1 and R-L when the elementary discourse unit fusion module is removed, because the direct concatenation of elementary discourse unit may bring redundancy into the summary and elementary discourse unit fusion can make the summary sentence more informative
  • We note that EDUSumsel+RL performs better than elementary discourse unit Sum with respect to R-2, perhaps because elementary discourse unit fusion may generate some fake information and need further improvement which will be our future work
  • EDUSum can fuse cross-sentence information and remedy the poor readability problem brought by elementary discourse unit
Methods
  • Empirical Methods in Natural Language

    Processing, pages 962–967, Brussels, Belgium. Association for Computational Linguistics.

    Yuxiang Wu and Baotian Hu. 2018.
  • Empirical Methods in Natural Language.
  • Processing, pages 962–967, Brussels, Belgium.
  • Association for Computational Linguistics.
  • Yuxiang Wu and Baotian Hu. 2018.
  • Learning to extract coherent summary via deep reinforcement learning.
  • In Thirty-Second AAAI Conference on Artificial Intelligence
Results
  • The authors compare the model with the state-of-the-art extractive and abstractive summarization methods.
  • EDUSumSameSent EDUSumgroup−1 EDUSumgroup−2 EDUSumgroup−3 EDUSum. Model Fast-Abs EDUSumsel+RL EDUSum reinforcement learning to extract and rewrite sentences.
  • Compared to Fast-Abs which is similar to EDUSum in model architecture, EDUSum achieves better performance with respect to the three metrics, showing EDU is more informative than sentence and appropriate to be the basic selection unit in summarization.
  • The authors design a model EDUSumsel+RL which is similar to EDUSum except that it does not include the EDU fusion module and directly concatenates the selected EDUs as a summary.
  • The authors note that EDUSumsel+RL performs better than EDU Sum with respect to R-2, perhaps because EDU fusion may generate some fake information and need further improvement which will be the future work
Conclusion
  • The authors choose EDU as the basic summary unit and propose a novel EDU based summarization model EDUSum.
  • The authors apply reinforcement learning to leverage EDU selection and EDU fusion for improving summarization performance.
  • With such a design, EDUSum can fuse cross-sentence information and remedy the poor readability problem brought by EDUs. Compared to previous work, this work has provided a feasible and effective method which makes full use of EDUs in summarization
Summary
  • Introduction:

    Abstractive summarization focuses on generating fluent and concise text from the original input document and has achieved considerable performance improvement with the rapid development of deep learning technology (See et al, 2017; Paulus et al, 2017; Celikyilmaz et al, 2018; Gehrmann et al, 2018).
  • The recently popular and practical paradigm usually generates summary sentences by independently compressing or rewriting each pre-extracted sentence, which is from the source documents (Chen and Bansal, 2018; Lebanoff et al, 2019).
  • A single document sentence usually cannot provide enough information that a summary sentence expresses, which is supported by the recent study of Lebanoff et al (2019).
  • The authors hope to seek a new summary composition unit which is more information-intensive and elementary than sentence
  • Methods:

    Empirical Methods in Natural Language

    Processing, pages 962–967, Brussels, Belgium. Association for Computational Linguistics.

    Yuxiang Wu and Baotian Hu. 2018.
  • Empirical Methods in Natural Language.
  • Processing, pages 962–967, Brussels, Belgium.
  • Association for Computational Linguistics.
  • Yuxiang Wu and Baotian Hu. 2018.
  • Learning to extract coherent summary via deep reinforcement learning.
  • In Thirty-Second AAAI Conference on Artificial Intelligence
  • Results:

    The authors compare the model with the state-of-the-art extractive and abstractive summarization methods.
  • EDUSumSameSent EDUSumgroup−1 EDUSumgroup−2 EDUSumgroup−3 EDUSum. Model Fast-Abs EDUSumsel+RL EDUSum reinforcement learning to extract and rewrite sentences.
  • Compared to Fast-Abs which is similar to EDUSum in model architecture, EDUSum achieves better performance with respect to the three metrics, showing EDU is more informative than sentence and appropriate to be the basic selection unit in summarization.
  • The authors design a model EDUSumsel+RL which is similar to EDUSum except that it does not include the EDU fusion module and directly concatenates the selected EDUs as a summary.
  • The authors note that EDUSumsel+RL performs better than EDU Sum with respect to R-2, perhaps because EDU fusion may generate some fake information and need further improvement which will be the future work
  • Conclusion:

    The authors choose EDU as the basic summary unit and propose a novel EDU based summarization model EDUSum.
  • The authors apply reinforcement learning to leverage EDU selection and EDU fusion for improving summarization performance.
  • With such a design, EDUSum can fuse cross-sentence information and remedy the poor readability problem brought by EDUs. Compared to previous work, this work has provided a feasible and effective method which makes full use of EDUs in summarization
Tables
  • Table1: Model Comparison ground-truth summary and the whole fused sentences as the reward for the final action that selects the stop label
  • Table2: Ablation Study on EDU Selection Module
  • Table3: Human Evaluation. The smaller value of the metric of the average rank, the better the performance
Download tables as Excel
Funding
  • This work was partially supported by National Key Research and Development Project (2019YFB1704002) and National Natural Science Foundation of China (61876009 and 61572049)
Reference
  • Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. arXiv preprint arXiv:1803.10357.
    Findings
  • Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 675–686, Melbourne, Australia. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 484–494, Berlin, Germany. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Angela Fan, David Grangier, and Michael Auli. 2017. Controllable abstractive summarization. arXiv preprint arXiv:1711.05217.
    Findings
  • Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. Bottom-up abstractive summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4098–4109, Brussels, Belgium. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 1693–1701. Curran Associates, Inc.
    Google ScholarLocate open access versionFindings
  • Sepp Hochreiter and Jurgen Schmidhuber. 199Long short-term memory. Neural Comput., 9(8):1735– 1780.
    Google ScholarLocate open access versionFindings
  • Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746–1751, Doha, Qatar. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. CoRR, abs/1412.6980.
    Findings
  • Logan Lebanoff, Kaiqiang Song, Franck Dernoncourt, Doo Soon Kim, Seokhwan Kim, Walter Chang, and Fei Liu. 2019. Scoring sentence singletons and pairs for abstractive summarization. arXiv preprint arXiv:1906.00077.
    Findings
  • Junyi Jessy Li, Kapil Thadani, and Amanda Stent. 2016. The role of discourse units in near-extractive summarization. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 137–147, Los Angeles. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Proc. ACL workshop on Text Summarization Branches Out, page 10.
    Google ScholarLocate open access versionFindings
  • Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text - Interdisciplinary Journal for the Study of Discourse, 8(3), 8(3):243– 281.
    Google ScholarLocate open access versionFindings
  • Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Ranking sentences for extractive summarization with reinforcement learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1747–1759, New Orleans, Louisiana. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304.
    Findings
  • Abigail See, Peter J. Liu, and Christopher D. Manning. 20Get to the point: Summarization with pointergenerator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073– 1083, Vancouver, Canada. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Yizhong Wang, Sujian Li, and Jingfeng Yang. 20Toward fast and accurate neural discourse segmentation. In Proceedings of the 2018 Conference on
    Google ScholarLocate open access versionFindings
  • Empirical Methods in Natural Language Processing, pages 962–967, Brussels, Belgium. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Yuxiang Wu and Baotian Hu. 2018. Learning to extract coherent summary via deep reinforcement learning. In Thirty-Second AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments