AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
While we experimented only with constituency syntax, SpanGCN may in principle be able to encode any kind of span structure, for example, coreference graphs, and can be used to produce linguistically-informed encoders for other NLP tasks rather than only semantic role labeling

Graph Convolutions over Constituent Trees for Syntax Aware Semantic Role Labeling

EMNLP 2020, pp.3915-3928, (2020)

被引用0|浏览175
下载 PDF 全文
引用
微博一下

摘要

Semantic role labeling (SRL) is the task of identifying predicates and labeling argument spans with semantic roles. Even though most semantic-role formalisms are built upon constituent syntax, and only syntactic constituents can be labeled as arguments (e.g., FrameNet and PropBank), all the recent work on syntax-aware SRL relies on depend...更多

代码

数据

0
简介
  • The task of semantic role labeling (SRL) consists of predicting the predicate-argument structure of a sentence.
  • The most popular resources for estimating SRL models are PropBank (Palmer et al, 2005) and FrameNet (Baker et al, 1998).
  • In both cases annotations are made on top of syntactic constituent structures.
  • Investors appeal to the CEO not to ARG0 AM-NEG.
  • ARG2 limit their access to sales data O
重点内容
  • The task of semantic role labeling (SRL) consists of predicting the predicate-argument structure of a sentence
  • Since SRL annotations are done on top of syntactic constituents,1 we argue that exploiting constituency syntax, rather than dependency one, is more natural and may yield more
  • Before comparing our full model to state-of-theart SRL systems, we show that our model genuinely benefits from incorporating syntactic information and motivate other modeling decisions
  • In this paper we introduced SpanGCN, a novel neural architecture encoding constituency syntax at the word level
  • Given that graph convolutional networks (GCNs) over dependency and constituency structure have access to very different information, it would be interesting to see in future work if combining two types of representations can lead to further improvements
  • Using ELMo as input word embeddings (EMB) is more effective than using it indirectly through predicted syntax (SYN), 85.9% vs. 85.7% F1
  • While we experimented only with constituency syntax, SpanGCN may in principle be able to encode any kind of span structure, for example, coreference graphs, and can be used to produce linguistically-informed encoders for other NLP tasks rather than only SRL
方法
  • 5.1 Data and setting

    The authors experimented on the CoNLL-2005 and CoNLL-2012 (OntoNotes) datasets, and used the CoNLL 2005 evaluation script for evaluation.
  • The authors applied the approach to FrameNet 1.5 with the data split of Das et al (2014) and followed the official evaluation set-up from the SemEval07 Task 19 on frame-semantic parsing (Baker et al, 2007).
  • The authors trained the self-attentive constituency parser of Kitaev and Klein (2018)5 on the training data of the CoNLL-2005 dataset and the authors parsed the development and test sets of CoNLL-2005 dataset.
  • For FrameNet, the authors parsed the entire corpus with the parser trained on the training set of CoNLL-2005
结果
  • Using ELMo as input word embeddings (EMB) is more effective than using it indirectly through predicted syntax (SYN), 85.9% vs. 85.7% F1.
  • When using both ELMo embeddings and the ELMo parser, the authors obtain even better scores 86.6% F1.
  • This result is 2.2% better than SpanGCN without ELMo and 0.65% better than the EMB model
结论
  • In this paper the authors introduced SpanGCN, a novel neural architecture encoding constituency syntax at the word level.
  • The authors applied SpanGCN to the semantic role labeling task, on PropBank and FrameNet. The authors can observe substantial improvements from using constituent syntax on both datasets, and in the realistic out-of-domain setting.
  • While the authors experimented only with constituency syntax, SpanGCN may in principle be able to encode any kind of span structure, for example, coreference graphs, and can be used to produce linguistically-informed encoders for other NLP tasks rather than only SRL
总结
  • Introduction:

    The task of semantic role labeling (SRL) consists of predicting the predicate-argument structure of a sentence.
  • The most popular resources for estimating SRL models are PropBank (Palmer et al, 2005) and FrameNet (Baker et al, 1998).
  • In both cases annotations are made on top of syntactic constituent structures.
  • Investors appeal to the CEO not to ARG0 AM-NEG.
  • ARG2 limit their access to sales data O
  • Methods:

    5.1 Data and setting

    The authors experimented on the CoNLL-2005 and CoNLL-2012 (OntoNotes) datasets, and used the CoNLL 2005 evaluation script for evaluation.
  • The authors applied the approach to FrameNet 1.5 with the data split of Das et al (2014) and followed the official evaluation set-up from the SemEval07 Task 19 on frame-semantic parsing (Baker et al, 2007).
  • The authors trained the self-attentive constituency parser of Kitaev and Klein (2018)5 on the training data of the CoNLL-2005 dataset and the authors parsed the development and test sets of CoNLL-2005 dataset.
  • For FrameNet, the authors parsed the entire corpus with the parser trained on the training set of CoNLL-2005
  • Results:

    Using ELMo as input word embeddings (EMB) is more effective than using it indirectly through predicted syntax (SYN), 85.9% vs. 85.7% F1.
  • When using both ELMo embeddings and the ELMo parser, the authors obtain even better scores 86.6% F1.
  • This result is 2.2% better than SpanGCN without ELMo and 0.65% better than the EMB model
  • Conclusion:

    In this paper the authors introduced SpanGCN, a novel neural architecture encoding constituency syntax at the word level.
  • The authors applied SpanGCN to the semantic role labeling task, on PropBank and FrameNet. The authors can observe substantial improvements from using constituent syntax on both datasets, and in the realistic out-of-domain setting.
  • While the authors experimented only with constituency syntax, SpanGCN may in principle be able to encode any kind of span structure, for example, coreference graphs, and can be used to produce linguistically-informed encoders for other NLP tasks rather than only SRL
表格
  • Table1: Results with predicted and gold syntax on the CoNLL-2005 development set
  • Table2: Precision, recall and F1 on the CoNLL-2005 development and test sets. † indicates syntactic models and ‡ indicates multi-task learning models
  • Table3: Precision, recall and F1 on the CoNLL-2012 test set. † indicates syntactic models and ‡ indicates multi-task learning models
  • Table4: Ablation results with ELMo information on CoNLL-2005 development set
  • Table5: Frame SRL results on the FrameNet 1.5 test set using gold frames. † indicates syntactic models and ‡ indicates multi-task learning models
  • Table6: Table 6
  • Table7: Precision, recall and F1 on the CoNLL-2012 development and test set. † indicates syntactic models and ‡ indicates multi-task learning models
Download tables as Excel
相关工作
  • Among earlier approaches to incorporating syntax in SRL, Socher et al (2013); Tai et al (2015) proposed recursive neural networks that encode constituency trees by recursively creating representations of constituents. There are two important differences with our approach. First, in our model the syntactic information in the constituents flows back to word representations. This may be achieved with their inside-outside versions (Le and Zuidema, 2014; Teng and Zhang, 2017) . Second, these previous model perform a global pass over the tree whereas GCNs take into account only small fragments of the graph. This may make GCNs more robust when using noisy predicted syntactic structures.
基金
  • The project was supported by the European Research Council (ERC StG BroadSem 678254), and the Dutch National Science Foundation (NWO VIDI 639.022.518)
研究对象与分析
datasets: 3
This approach directly encodes into word representation information about boundaries and syntactic labels of constituents and also provides information about their neighbourhood in the constituent structure. We show effectiveness of our approach on three datasets: CoNLL-2005 (Carreras and Marquez, 2005) and CoNLL-2012 (Pradhan et al, 2012) with PropBank-style (Palmer et al, 2005) annotation and on FrameNet 1.5 (Baker et al, 1998). SpanGCNs may be beneficial in other NLP tasks, where neural sentence encoders are already effective and syntactic structure can provide a useful inductive bias

引用论文
  • Lei Jimmy Ba, Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. CoRR, abs/1607.06450.
    Findings
  • Collin F. Baker, Michael Ellsworth, and Katrin Erk. 2007. Semeval-2007 task 19: Frame semantic structure extraction. In Proceedings of SemEval@ACL, pages 99–104.
    Google ScholarLocate open access versionFindings
  • Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The berkeley framenet project. In Proceedings of COLING-ACL, pages 86–90.
    Google ScholarLocate open access versionFindings
  • Xavier Carreras and Lluıs Marquez. 2005. Introduction to the conll-2005 shared task: Semantic role labeling. In Proceedings of CoNLL, pages 152–164.
    Google ScholarLocate open access versionFindings
  • Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of NAACL-HLT, pages 93–98.
    Google ScholarLocate open access versionFindings
  • Dipanjan Das, Desai Chen, Andre F. T. Martins, Nathan Schneider, and Noah A. Smith. 2014. Frame-semantic parsing. Computational Linguistics, 40(1):9–56.
    Google ScholarLocate open access versionFindings
  • Li Dong and Mirella Lapata. 2016. Language to logical form with neural attention. In Proceedings of ACL.
    Google ScholarLocate open access versionFindings
  • Nicholas FitzGerald, Oscar Tackstrom, Kuzman Ganchev, and Dipanjan Das. 2015. Semantic role labeling with neural network factors. In Proceedings of EMNLP.
    Google ScholarLocate open access versionFindings
  • Yarin Gal and Zoubin Ghahramani. 2016. A theoretically grounded application of dropout in recurrent neural networks. In Proceedings of NIPS, pages 1019–1027.
    Google ScholarLocate open access versionFindings
  • Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational linguistics, 28(3):245–288.
    Google ScholarLocate open access versionFindings
  • Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212.
    Findings
  • Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850.
    Findings
  • Luheng He, Kenton Lee, Omer Levy, and Luke Zettlemoyer. 2018. Jointly predicting predicates and arguments in neural semantic role labeling. In Proceedings of ACL, pages 364–369.
    Google ScholarLocate open access versionFindings
  • Luheng He, Kenton Lee, Mike Lewis, and Luke Zettlemoyer. 2017. Deep semantic role labeling: What works and what’s next. In Proceedings of ACL, pages 473–483.
    Google ScholarLocate open access versionFindings
  • John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of NAACL-HLT, pages 4129–4138.
    Google ScholarLocate open access versionFindings
  • Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780.
    Google ScholarLocate open access versionFindings
  • Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR.
    Google ScholarLocate open access versionFindings
  • Thomas N. Kipf and Max Welling. 2017. Semisupervised classification with graph convolutional networks. In Proceedings of ICLR.
    Google ScholarLocate open access versionFindings
  • Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In Proceedings of ACL, pages 2675–2685.
    Google ScholarLocate open access versionFindings
  • John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML, pages 282– 289.
    Google ScholarLocate open access versionFindings
  • Phong Le and Willem Zuidema. 2014. The insideoutside recursive neural network model for dependency parsing. In Proceedings of EMNLP.
    Google ScholarLocate open access versionFindings
  • Zuchao Li, Shexia He, Hai Zhao, Yiqing Zhang, Zhuosheng Zhang, Xi Zhou, and Xiang Zhou. 2019. Dependency or span, end-to-end uniform semantic role labeling. CoRR, abs/1901.05280.
    Findings
  • Diego Marcheggiani, Anton Frolov, and Ivan Titov. 2017. A simple and accurate syntax-agnostic neural model for dependency-based semantic role labeling. In Proceedings of CoNLL.
    Google ScholarLocate open access versionFindings
  • Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In Proceedings of EMNLP, pages 1506–1515.
    Google ScholarLocate open access versionFindings
  • Alessandro Moschitti, Daniele Pighin, and Roberto Basili. 2008. Tree kernels for semantic role labeling. Computational Linguistics, 34(2):193–224.
    Google ScholarLocate open access versionFindings
  • Jason Naradowsky, Sebastian Riedel, and David A Smith. 2012. Improving nlp through marginalization of hidden syntactic structure. In Proceedings of EMNLP.
    Google ScholarLocate open access versionFindings
  • Hiroki Ouchi, Hiroyuki Shindo, and Yuji Matsumoto. 2018. A span selection model for semantic role labeling. In Proceedings of EMNLP, pages 1630– 1642.
    Google ScholarLocate open access versionFindings
  • Martha Palmer, Paul Kingsbury, and Daniel Gildea. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106.
    Google ScholarLocate open access versionFindings
  • Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of EMNLP, pages 1532– 1543.
    Google ScholarLocate open access versionFindings
  • Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. Deep contextualized word representations. In Proceedings of NAACL-HLT, pages 2227–2237.
    Google ScholarLocate open access versionFindings
  • Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018b. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of EMNLP, pages 1499–1509.
    Google ScholarLocate open access versionFindings
  • Sameer Pradhan, Kadri Hacioglu, Wayne H. Ward, James H. Martin, and Daniel Jurafsky. 2005. Semantic role chunking combining complementary syntactic views. In Proceedings of CoNLL.
    Google ScholarLocate open access versionFindings
  • Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Proceedings of EMNLP-CoNLL, pages 1–40.
    Google ScholarLocate open access versionFindings
  • Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics, 34(2):257–287.
    Google ScholarLocate open access versionFindings
  • Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In Proceedings of ACL.
    Google ScholarLocate open access versionFindings
  • Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP.
    Google ScholarLocate open access versionFindings
  • Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958.
    Google ScholarLocate open access versionFindings
  • Rupesh Kumar Srivastava, Klaus Greff, and Jurgen Schmidhuber. 2015. Training very deep networks. In Proceedings of NIPS, pages 2377–2385.
    Google ScholarLocate open access versionFindings
  • Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. 2018. Linguistically-informed self-attention for semantic role labeling. In Proceedings of EMNLP, pages 5027–5038.
    Google ScholarLocate open access versionFindings
  • Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluıs Marquez, and Joakim Nivre. 2008. The conll 2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of CoNLL.
    Google ScholarLocate open access versionFindings
  • Swabha Swayamdipta, Sam Thomson, Kenton Lee, Luke Zettlemoyer, Chris Dyer, and Noah A. Smith. 2018. Syntactic scaffolds for semantic structures. In Proceedings of EMNLP, pages 3772–3782.
    Google ScholarLocate open access versionFindings
  • Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of ACL-IJCNLP, pages 1556–1566.
    Google ScholarLocate open access versionFindings
  • Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, and Xiaodong Shi. 2018. Deep semantic role labeling with self-attention. In Proceedings of AAAI, pages 4929–4936.
    Google ScholarLocate open access versionFindings
  • Zhiyang Teng and Yue Zhang. 2017. Head-lexicalized bidirectional tree lstms. TACL, 5:163–177.
    Google ScholarLocate open access versionFindings
  • Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. 2019. What do you learn from context? Probing for sentence structure in contextualized word representations. In Proceedings of ICLR.
    Google ScholarLocate open access versionFindings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NIPS, pages 6000– 6010.
    Google ScholarLocate open access versionFindings
  • Bishan Yang and Tom M. Mitchell. 2017. A joint sequential and relational model for frame-semantic parsing. In Proceedings of EMNLP, pages 1247– 1256.
    Google ScholarLocate open access versionFindings
  • Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In Proceedings of ACL.
    Google ScholarLocate open access versionFindings
  • He et al. (2017) He et al. (2018) Tan et al. (2018) Ouchi et al. (2018) Strubell et al. (2018)†‡ SpanGCN†
    Google ScholarLocate open access versionFindings
  • He et al. (2018) Li et al. (2019) Ouchi et al. (2018) SpanGCN†
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科