AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We present a novel distant supervision approach to predict the discourse-structure and nuclearity for documents of arbitrary length solely using document-level sentiment information

MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision

EMNLP 2020, pp.7442-7457, (2020)

Cited by: 0|Views161
Full Text
Bibtex
Weibo

Abstract

The lack of large and diverse discourse treebanks hinders the application of data-driven approaches, such as deep-learning, to RST-style discourse parsing. In this work, we present a novel scalable methodology to automatically generate discourse treebanks using distant supervision from sentiment annotated datasets, creating and publishing...More

Code:

Data:

0
Introduction
  • Discourse parsing is an important Natural Language Processing (NLP) task, aiming to uncover the hidden structure underlying coherent documents, as described by theories of discourse like Rhetorical Structure Theory (RST) (Mann and Thompson, 1988) or PDTB (Prasad et al, 2008).
  • The authors' recent efforts using distant supervision from sentiment to generate large-scale discourse treebanks have already partly addressed this dire situation (Huber and Carenini, 2019), the previously proposed solution is still limited in: (i) Scope, by only building the RST constituency structure without nuclearity and relation labels; and (ii) Applicability, by relying on a non-scalable CKY solution, which cannot be applied to many real-world datasets with especially long documents
Highlights
  • Discourse parsing is an important Natural Language Processing (NLP) task, aiming to uncover the hidden structure underlying coherent documents, as described by theories of discourse like Rhetorical Structure Theory (RST) (Mann and Thompson, 1988) or PDTB (Prasad et al, 2008)
  • Our recent efforts using distant supervision from sentiment to generate large-scale discourse treebanks have already partly addressed this dire situation (Huber and Carenini, 2019), the previously proposed solution is still limited in: (i) Scope, by only building the RST constituency structure without nuclearity and relation labels; and (ii) Applicability, by relying on a non-scalable CKY solution, which cannot be applied to many real-world datasets with especially long documents
  • Inspired by the recent success in applying beamsearch to enhance the scalability of multiple NLP parsing tasks (Mabona et al, 2019; Fried et al, 2017; Dyer et al, 2016; Vinyals et al, 2015), we propose a novel heuristic beam-search approach that can automatically generate discourse trees containing structure- and nuclearity-attributes for documents of arbitrary length
  • We evaluate the individual components of our system by showing an ablation study in Table 4, starting with the performance of the discourse parser trained with MEGA-DT-Base, a treebank generated with the standard beam-search approach and without integrating nuclearity
  • We present a novel distant supervision approach to predict the discourse-structure and nuclearity for documents of arbitrary length solely using document-level sentiment information
  • Our results on the challenging inter-domain discourse-structure and -nuclearity prediction task strongly suggests that the heuristic approach taken (1) enhances the structure prediction task through more diversity in the early-stage tree selection, (2) allows us to effectively predict nuclearity and (3) helps to significantly reduce the complexity of the unrestricted CKY approach to scale for arbitrary length documents
Results
  • The authors train the discourse parser by Wang et al (2017)3 on the newly generated MEGA-DT corpus as well as the Yelp13-DT and the original RST-DT and

    2www.nltk.org/api/nltk.tokenize.html 3www.github.com/yizhongw/StageDP/ Approach

    Right Branching Left Branching Hier.
  • The authors train the discourse parser by Wang et al (2017)3 on the newly generated MEGA-DT corpus as well as the Yelp13-DT and the original RST-DT and.
  • Right Branching Left Branching Hier.
  • Right Branching Hier.
  • Left Branching Majority Class.
  • DPLP(2014)* gCRF(2014)* CODRA(2015)* Li(2016)* Two-Stage(2017) Yu(2018).
  • Two-StageRST-DT Two-StageInstr-DT Two-StageYelp13-DT(2019) Two-StageMEGA-DT Human (2017) Structure RST-DT Instr-DT
Conclusion
  • Conclusions and Future Work

    In this work, the authors present a novel distant supervision approach to predict the discourse-structure and nuclearity for documents of arbitrary length solely using document-level sentiment information.
  • As a case study for the effectiveness of the approach, the authors annotate and publish the MEGA-DT corpus as a high quality RSTstyle discourse treebank, which has been shown to outperform previously proposed discourse treebanks on most tasks of inter-domain discourse parsing
  • This suggests that parsers trained on the MEGADT corpus should be used to derive discourse trees in target domains where no gold-labeled data is available
Tables
  • Table1: Upper-bounds for growth of spatial complexity using different beam sizes and unconstrained CKY (∞), assuming 1Byte per unit in memory. KB = 103, MB = 106, GB = 109, PB = 1015, SB = 1054
  • Table2: Treebank size and distribution (*calculated on the training set)
  • Table3: Results of the micro-averaged precision measure using the original Parseval method (Par.) and RST Parseval (R-Par.). Inter-domain subscripts identify the training set. Inter-domain results averaged over 10 independent runs. Models with stochastic components are averaged over 3 distinct generation processes. The best performance per sub-table is bold. * Results taken from <a class="ref-link" id="cMorey_et+al_2017_a" href="#rMorey_et+al_2017_a">Morey et al (2017</a>), † statistically significant with p-value ≤ .05 to the best inter-domain baseline (Bonferroni adjusted), – non-published values, × not feasible combinations
  • Table4: Ablation study showing the influence of nuclearity and stochasticity on the overall performance, measured as the micro-average precision using original Parseval (Par.) and RST Parseval (R-Par.). Results averaged over 10 runs (using 3 distinct generation processes if a stochastic components is included). The best performance is bold
  • Table5: Confusion Matrices for the model trained on MEGA-DT, evaluated on RST-DT (top) and Instr-DT (bottom). Left: Original Parseval, Right: RST-Parseval identifying the over-prediction of the N-N class, especially for gold-label N-S nuclearities. Further, we frequently misclassify the gold-label N-S nuclearity class as S-N. Lastly, we present an additional qualitative analysis in Appendix A to investigate the strength and potential weaknesses of trees in MEGA-DT. We therefore show three randomly selected trees that closely/poorly reflect the authors gold-label sentiment respectively (see Figure 4 for a teaser). In general, the qualitative analysis shows that trees in MEGA-DT are non-trivial, reasonably balanced, strongly linked to the EDU-level sentiment and mostly well-aligned with meaningful discourse-structures
Download tables as Excel
Related work
  • The most closely related line of work is RST-style discourse parsing, with the goal to obtain a complete discourse tree, including structure, nuclearity and relations. Based on the observation that these three aspects are correlated, most previous work has explored models to learn them jointly (e.g., Joty et al (2015); Ji and Eisenstein (2014); Yu et al (2018)). However, while this strategy seems intuitive, the state-of-the-art (SOTA) system on structure-prediction by Wang et al (2017) applies a rather different strategy, first jointly predicting structure and nuclearity and then subsequently pre-

    dicting relations. The main motivation behind this separation is the large number of possible output classes when predicting these three aspects together. The success of the system by Wang et al (2017) on the widely used RST-DT corpus inspires us to also learn structure and nuclearity jointly, rather than combining all three aspects.

    The second line of related work infers finegrained information from coarse-grained supervision signals using machine learning. Due to the lack of annotated data in many domains and for many real-world tasks, methods to automatically generate reliable, fine-grained data-labels have been explored for many years. One promising approach in this area is Multiple Instance Learning (MIL) (Keeler et al, 1991). The general task of MIL is to retrieve fine-grained information (called instance-labels) from high-level supervision (called bag-labels), using correlations of discriminative features within and between bags to predict labels for instances. With the recent rise of deep-learning, neural MIL approaches have also been proposed (Angelidis and Lapata, 2018).
Funding
  • This research was supported by the Language & Speech Innovation Lab of Cloud BU, Huawei Technologies Co., Ltd
Study subjects and analysis
documents: 100
A key limitation for further research in RST-style discourse parsing is the scarcity of training data. Only a few human annotated discourse treebanks exist, each only containing a few hundred documents. Although our recent efforts using distant supervision from sentiment to generate large-scale discourse treebanks have already partly addressed this dire situation (Huber and Carenini, 2019), the previously proposed solution is still limited in: (i) Scope, by only building the RST constituency structure without nuclearity and relation labels; and (ii) Applicability, by relying on a non-scalable CKY solution, which cannot be applied to many real-world datasets with especially long documents

documents: 10000
4.3 Preliminary Evaluation. We run a set of preliminary evaluations on a randomly selected subset containing 10,000 documents from the Yelp’13 dataset. In general, the preliminary evaluation suggests that (1) A beamsize of 10 delivers the best trade-off between computational complexity and performance (out of {1, 5, 10, 50, 100}), when tested according to the distance between gold-label sentiment and model prediction

Reference
  • Stefanos Angelidis and Mirella Lapata. 2018. Multiple instance learning networks for fine-grained sentiment analysis. Transactions of the Association for Computational Linguistics, 6:17–31.
    Google ScholarLocate open access versionFindings
  • Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from rst discourse parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2212–2218.
    Google ScholarLocate open access versionFindings
  • Lynn Carlson, Mary Ellen Okurowski, and Daniel Marcu. 2002. RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania.
    Google ScholarFindings
  • Tuhin Chakrabarty, Christopher Hidey, Smaranda Muresan, Kathleen McKeown, and Alyssa Hwang. 2019. Ampersand: Argument mining for persuasive online discussions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pages 2926–2936.
    Google ScholarLocate open access versionFindings
  • Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 615–621, New Orleans, Louisiana. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
    Findings
  • Qiming Diao, Minghui Qiu, Chao-Yuan Wu, Alexander J Smola, Jing Jiang, and Chong Wang. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 193– 202. ACM.
    Google ScholarLocate open access versionFindings
  • Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A Smith. 2016. Recurrent neural network grammars. arXiv preprint arXiv:1602.07776.
    Findings
  • Vanessa Wei Feng and Graeme Hirst. 2014. A lineartime bottom-up discourse parser with constraints and post-editing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 511–521.
    Google ScholarLocate open access versionFindings
  • Elisa Ferracane, Greg Durrett, Junyi Jessy Li, and Katrin Erk. 2019. Evaluating discourse in structured text representations. arXiv preprint arXiv:1906.01472.
    Findings
  • Daniel Fried, Mitchell Stern, and Dan Klein. 2017. Improving neural parsing by disentangling model combination and reranking effects. arXiv preprint arXiv:1707.03058.
    Findings
  • Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1602–1613.
    Google ScholarLocate open access versionFindings
  • Alexander Hogenboom, Flavius Frasincar, Franciska De Jong, and Uzay Kaymak. 2015. Using rhetorical structure in sentiment analysis. Commun. ACM, 58(7):69–77.
    Google ScholarLocate open access versionFindings
  • Patrick Huber and Giuseppe Carenini. 2019. Predicting discourse structure using distant supervision from sentiment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pages 2306–2316.
    Google ScholarLocate open access versionFindings
  • Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 13–24.
    Google ScholarLocate open access versionFindings
  • Yangfeng Ji and Noah A Smith. 2017. Neural discourse structure for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 996–1005.
    Google ScholarLocate open access versionFindings
  • Shafiq Joty, Giuseppe Carenini, and Raymond T Ng. 2015. Codra: A novel discriminative framework for rhetorical analysis. Computational Linguistics, 41(3).
    Google ScholarLocate open access versionFindings
  • Dan Jurafsky and James H Martin. 2014. Speech and language processing, volume 3. Pearson London.
    Google ScholarLocate open access versionFindings
  • Hamid Karimi and Jiliang Tang. 20Learning hierarchical discourse-level structure for fake news detection. arXiv preprint arXiv:1903.07389.
    Findings
  • James D Keeler, David E Rumelhart, and Wee Kheng Leow. 1991. Integrated segmentation and recognition of hand-printed numerals. In Advances in neural information processing systems, pages 557–563.
    Google ScholarLocate open access versionFindings
  • Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 362–371.
    Google ScholarLocate open access versionFindings
  • Yang Liu, Ivan Titov, and Mirella Lapata. 2019. Single document summarization as tree induction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1745–1755.
    Google ScholarLocate open access versionFindings
  • Amandla Mabona, Laura Rimell, Stephen Clark, and Andreas Vlachos. 2019. Neural generative rhetorical structure parsing. arXiv preprint arXiv:1909.11049.
    Findings
  • William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243–281.
    Google ScholarLocate open access versionFindings
  • Daniel Marcu. 2000. The theory and practice of discourse parsing and summarization. MIT press.
    Google ScholarFindings
  • Mathieu Morey, Philippe Muller, and Nicholas Asher. 2017. How much progress have we made on rst discourse parsing? a replication study of recent results on the rst-dt. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1319–1324.
    Google ScholarLocate open access versionFindings
  • Mathieu Morey, Philippe Muller, and Nicholas Asher. 2018. A dependency perspective on rst discourse parsing and evaluation. Computational Linguistics, 44(2):197–235.
    Google ScholarLocate open access versionFindings
  • Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar GuI‡lcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 280–290, Berlin, Germany. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • American Chapter of the Association for Computational Linguistics, pages 566–574. Association for Computational Linguistics.
    Google ScholarFindings
  • Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1422–1432.
    Google ScholarLocate open access versionFindings
  • Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Advances in neural information processing systems, pages 2773–2781.
    Google ScholarLocate open access versionFindings
  • Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 184–188.
    Google ScholarLocate open access versionFindings
  • Nan Yu, Meishan Zhang, and Guohong Fu. 2018. Transition-based neural rst parsing with implicit syntax features. In Proceedings of the 27th International Conference on Computational Linguistics, pages 559–570.
    Google ScholarLocate open access versionFindings
  • Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems, pages 649–657.
    Google ScholarLocate open access versionFindings
  • Bita Nejat, Giuseppe Carenini, and Raymond Ng. 2017. Exploring joint neural model for sentence level discourse parsing and sentiment analysis. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 289–298.
    Google ScholarLocate open access versionFindings
  • David L Poole and Alan K Mackworth. 2010. Artificial Intelligence: foundations of computational agents. Cambridge University Press.
    Google ScholarFindings
  • Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. LREC.
    Google ScholarLocate open access versionFindings
  • Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822.
    Findings
  • Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
    Findings
  • Vighnesh Shiv and Chris Quirk. 2019. Novel positional encodings to enable tree-based transformers. In Advances in Neural Information Processing Systems, pages 12058–12068.
    Google ScholarLocate open access versionFindings
  • Rajen Subba and Barbara Di Eugenio. 2009. An effective discourse parser that uses rich linguistic information. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North
    Google ScholarLocate open access versionFindings
Author
Patrick Huber
Patrick Huber
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科