AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
Phrase alignment generally adheres to compositionality, in which a phrase pair is aligned based on the alignments of their child phrases

Compositional Phrase Alignment and Beyond

EMNLP 2020, pp.1611-1623, (2020)

被引用0|浏览168
下载 PDF 全文
引用
微博一下

摘要

Phrase alignment is the basis for modelling sentence pair interactions, such as paraphrase and textual entailment recognition. Most phrase alignments are compositional processes such that an alignment of a phrase pair is constructed based on the alignments of their child phrases. Nonetheless, studies have revealed that non-compositional a...更多

代码

数据

0
简介
  • Phrase alignment is a fundamental problem in modelling the interactions between a pair of sentences, such as paraphrase identification, textual entailment recognition, and question answering (Das and Smith, 2009; Heilman and Smith, 2010; Wang and Manning, 2010).
  • The alignment of τ1s and τ1t is non-compositional in relation to the alignment of τ2s and τ2t; τ1t and τ2t are siblings, τ1s is not a sibling of τ2s, i.e. not in the scope of the parent node of τ2s
  • To treat such a long-distance correspondence in non-compositional alignment, one has to consider candidate phrases outside the local scope and potentially the entire sentence
重点内容
  • Phrase alignment is a fundamental problem in modelling the interactions between a pair of sentences, such as paraphrase identification, textual entailment recognition, and question answering (Das and Smith, 2009; Heilman and Smith, 2010; Wang and Manning, 2010)
  • Phrase alignment generally adheres to compositionality, in which a phrase pair is aligned based on the alignments of their child phrases
  • Our experimental results indicate that the proposed method achieves 95.7% of the alignment quality of trained human annotators for phrase alignment in paraphrase sentence pairs
  • We extended the Syntactic Phrase Alignment Dataset for Evaluation (SPADE) (Arase and Tsujii, 2018), creating the Extended Syntactic Phrase Alignment DAtaset (ESPADA)
  • Our method significantly outperforms that used in a previous study and achieves a performance competitive with that of experienced human annotators
  • We investigated non-compositional alignments produced by bidirectional encoder representations from transformers (BERT)+SimMatrix+CTED with postprocessing
方法
  • As the comparison state-of-the-art syntactic phrase alignment method, the authors used Arase and Tsujii (2017).
  • ALIR and ALIP (%) 60 ALIR
  • It encourages a null alignment in CTED when there is a more similar phrase beyond the local scope.
  • A large λ∅ confuses the method by allowing a larger number of possible alignments
  • Both situations are harmful, but the former has a larger impact.
  • But the former has a larger impact
  • This is because the constraint of CTED only allows a legitimate set of phrase alignments, which effectively prunes away incorrect alignments.
结果
  • 6.1 Overall Results

    Table 3 compares the methods’ performance. BERT+SimMatrix+CTED (last row) includes the full feature set; it transforms the phrase representation using SimMatrix representation and aligns phrases using CTED.
  • BERT+SimMatrix+CTED includes the full feature set; it transforms the phrase representation using SimMatrix representation and aligns phrases using CTED.
  • This method performed the best overall, achieving an ALIF score of 87.4% with post-processing.
  • The authors found that 0.1% of alignment pairs did not satisfy the monotonicity condition and 1.2% of alignment triplets did not satisfy the familiness condition
  • These non-compositional alignments cover 3.5% and 23.2% of those of the gold standard that did not satisfy the monotonicity and familiness conditions, respectively
结论
  • In contrast to previous methods, ours can align phrases in paraphrasal sentence pairs and in partially paraphrasal pairs.
  • The authors intend to expand the method to conduct forest alignments for making it robust against parsing errors, which are inevitable in handling large corpora.
  • As the method does not restrict input to syntactic trees but only assumes tree structures with arbitrary numbering as input, the authors intend to try alignments of chunk-based trees, which is desirable for applications that process text fragments, e.g. those that perform information extraction
总结
  • Introduction:

    Phrase alignment is a fundamental problem in modelling the interactions between a pair of sentences, such as paraphrase identification, textual entailment recognition, and question answering (Das and Smith, 2009; Heilman and Smith, 2010; Wang and Manning, 2010).
  • The alignment of τ1s and τ1t is non-compositional in relation to the alignment of τ2s and τ2t; τ1t and τ2t are siblings, τ1s is not a sibling of τ2s, i.e. not in the scope of the parent node of τ2s
  • To treat such a long-distance correspondence in non-compositional alignment, one has to consider candidate phrases outside the local scope and potentially the entire sentence
  • Methods:

    As the comparison state-of-the-art syntactic phrase alignment method, the authors used Arase and Tsujii (2017).
  • ALIR and ALIP (%) 60 ALIR
  • It encourages a null alignment in CTED when there is a more similar phrase beyond the local scope.
  • A large λ∅ confuses the method by allowing a larger number of possible alignments
  • Both situations are harmful, but the former has a larger impact.
  • But the former has a larger impact
  • This is because the constraint of CTED only allows a legitimate set of phrase alignments, which effectively prunes away incorrect alignments.
  • Results:

    6.1 Overall Results

    Table 3 compares the methods’ performance. BERT+SimMatrix+CTED (last row) includes the full feature set; it transforms the phrase representation using SimMatrix representation and aligns phrases using CTED.
  • BERT+SimMatrix+CTED includes the full feature set; it transforms the phrase representation using SimMatrix representation and aligns phrases using CTED.
  • This method performed the best overall, achieving an ALIF score of 87.4% with post-processing.
  • The authors found that 0.1% of alignment pairs did not satisfy the monotonicity condition and 1.2% of alignment triplets did not satisfy the familiness condition
  • These non-compositional alignments cover 3.5% and 23.2% of those of the gold standard that did not satisfy the monotonicity and familiness conditions, respectively
  • Conclusion:

    In contrast to previous methods, ours can align phrases in paraphrasal sentence pairs and in partially paraphrasal pairs.
  • The authors intend to expand the method to conduct forest alignments for making it robust against parsing errors, which are inevitable in handling large corpora.
  • As the method does not restrict input to syntactic trees but only assumes tree structures with arbitrary numbering as input, the authors intend to try alignments of chunk-based trees, which is desirable for applications that process text fragments, e.g. those that perform information extraction
表格
  • Table1: Statistics for ESPADA and SPADE (‘#’ stands for ‘number of’)
  • Table2: Human performance the familiness condition in alignments agreed upon by at least two annotators. Note that the monotonicity and familiness conditions are defined on relations of alignment pairs and triples, respectively; hence, these percentages do not mean that these percentages of alignments are non-compositional
  • Table3: ALIR, ALIP, and ALIF scores with 95% confidence intervals
  • Table4: ALIR, ALIP, and ALIF scores with feature-based approaches pensated for by CTED. In contrast, the smaller improvements on ELMo and BERT imply that they obtain such structural information through their masked language model training. This result is consistent with previous studies (<a class="ref-link" id="cHewitt_2019_a" href="#rHewitt_2019_a">Hewitt and Manning, 2019</a>; <a class="ref-link" id="cJawahar_et+al_2019_a" href="#rJawahar_et+al_2019_a">Jawahar et al, 2019</a>; Reif et al, 2019) that confirmed that BERT learns syntactic structures
  • Table5: ALIR, ALIP, and ALIF scores for our phrase representation model when applied to feature-based models (‘BERT w/o FT’ stands for ‘BERT without fine-tuning’)
Download tables as Excel
相关工作
  • Related Work resentations

    In this study, we propose dedicated phrase representations for the alignment problem. Before contextualised representation, studies considered word alignment distributions for modelling semantic interactions between a pair of sentences (He and Lin, 2016; Parikh et al, 2016; Chen et al, 2017). We agree with their intuition that the pairwise similarities alone are not good enough to define the cost of alignment. In case there are other similar phrases, their pairwise similarities have to be properly adjusted. This adjustment is crucial for treating non-compositional global alignment.

    2.1 Tree Mapping and Phrase Alignment

    Ordered tree mapping has been employed to estimate the similarity of a pair of sentences for its ability to align syntactic trees (Punyakanok et al, 2004; Alabbas and Ramsay, 2013; Yao et al, 2013; McCaffery and Nederhof, 2016). However, it is too restrictive in that the order of the aligned phrases in the sentences must be the same. Previous studies extended the algorithm to adapt the edit costs (Bernard et al, 2008; Mehdad, 2009; Alabbas and Ramsay, 2013) and edit operations (Heilman and Smith, 2010; Wang and Manning, 2010) to specific tasks. In contrast, the unordered tree mapping that we employ in this study is sufficiently flexible to assure identification of optimal compositional phrase alignments.
基金
  • This work was supported by JST, ACT-I, Grant Number JPMJPR16U2, Japan
研究对象与分析
sentence pairs: 1916
We extended the Syntactic Phrase Alignment Dataset for Evaluation (SPADE) (Arase and Tsujii, 2018), creating the Extended Syntactic Phrase Alignment DAtaset (ESPADA). Following the same annotation scheme, we annotated 1, 916 sentence pairs sampled from NIST OpenMT4 corpora. ESPADA is now the largest annotation corpus for this problem and will be released by the Linguistic Data Consortium (LDC) soon

引用论文
  • Maytham Alabbas and Allan Ramsay. 2013. Optimising tree edit distance with subtrees for textual entailment. In Proceedings of International Conference Recent Advances in Natural Language Processing, pages 9–17.
    Google ScholarLocate open access versionFindings
  • Yuki Arase and Junichi Tsujii. 2017. Monolingual phrase alignment on parse forests. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1–11.
    Google ScholarLocate open access versionFindings
  • Yuki Arase and Junichi Tsujii. 2018. SPADE: Evaluation dataset for monolingual phrase alignment. In Proceedings of the Language Resources and Evaluation Conference (LREC).
    Google ScholarLocate open access versionFindings
  • Yuki Arase and Jun’ichi Tsujii. 2019. Transfer finetuning: A BERT case study. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5392–5403.
    Google ScholarLocate open access versionFindings
  • Marc Bernard, Laurent Boyer, Amaury Habrard, and Marc Sebban. 2008. Learning probabilistic models of tree edit distance. Pattern Recognition, 41(8):2611–2629.
    Google ScholarFindings
  • Philip Bille. 2005. A survey on tree edit distance and related problems. Theoretical Computer Science, 337(1—3):217––239.
    Google ScholarLocate open access versionFindings
  • Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 201Enriching word vectors with subword information. Transactions of the Association of Computational Linguistics (TACL), 5:135– 146.
    Google ScholarLocate open access versionFindings
  • David Burkett, John Blitzer, and Dan Klein. 2010. Joint parsing and alignment with weakly synchronized grammars. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 127–135.
    Google ScholarLocate open access versionFindings
  • Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for natural language inference. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 1657–1668.
    Google ScholarLocate open access versionFindings
  • Do Kook Choe and David McClosky. 2015. Parsing paraphrases with joint inference. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP), pages 1223–1233.
    Google ScholarLocate open access versionFindings
  • Dipanjan Das and Noah A. Smith. 2009. Paraphrase identification as probabilistic quasi-synchronous recognition. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACLIJCNLP), pages 468–476.
    Google ScholarLocate open access versionFindings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186.
    Google ScholarLocate open access versionFindings
  • Bill Dolan, Chris Quirk, and Chris Brockett. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 350–356.
    Google ScholarLocate open access versionFindings
  • Bradley Efron and R.J. Tibshirani. 1994. An Introduction to the Bootstrap. Chapman and Hall/CRC.
    Google ScholarFindings
  • Hua He and Jimmy Lin. 2016. Pairwise word interaction modeling with deep neural networks for semantic similarity measurement. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 937–948.
    Google ScholarLocate open access versionFindings
  • Michael Heilman and Noah A. Smith. 2010. Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACLHLT), pages 1011–1019.
    Google ScholarLocate open access versionFindings
  • John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4129–4138.
    Google ScholarLocate open access versionFindings
  • Ganesh Jawahar, Benoıt Sagot, and Djame Seddah. 2019. What does BERT learn about the structure of language? In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 3651–3657.
    Google ScholarLocate open access versionFindings
  • Wuwei Lan, Siyu Qiu, Hua He, and Wei Xu. 2017. A continuously growing dataset of sentential paraphrases. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1235–1245.
    Google ScholarLocate open access versionFindings
  • Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proceedings of the International Conference on Learning Representations (ICLR).
    Google ScholarLocate open access versionFindings
  • Pascual Martınez-Gomez and Yusuke Miyao. 2016. Rule extraction for tree-to-tree transducers by cost minimization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 12–22.
    Google ScholarLocate open access versionFindings
  • Martin McCaffery and Mark-Jan Nederhof. 2016. DTED: Evaluation of machine translation structure using dependency parsing and tree edit distance. In Proceedings of the Workshop on Statistical Machine Translation (WMT), pages 491–498.
    Google ScholarLocate open access versionFindings
  • Yashar Mehdad. 2009. Automatic cost estimation for tree edit distance using particle swarm optimization. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP), pages 289–292.
    Google ScholarLocate open access versionFindings
  • Ankur Parikh, Oscar Tackstrom, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2249– 2255.
    Google ScholarLocate open access versionFindings
  • Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 2227– 2237.
    Google ScholarLocate open access versionFindings
  • Vasin Punyakanok, Dan Roth, and Wen tau Yih. 2004. Mapping dependencies trees: An application to question answering. International Symposium on Artificial Intelligence and Mathematics (Special Session).
    Google ScholarLocate open access versionFindings
  • Emily Reif, Ann Yuan, Martin Wattenberg, Fernanda B Viegas, Andy Coenen, Adam Pearce, and Been Kim. 2019. Visualizing and measuring the geometry of BERT. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS), pages 8594–8603.
    Google ScholarLocate open access versionFindings
  • David Smith and Jason Eisner. 2006. Quasisynchronous grammars: Alignment by soft projection of syntactic dependencies. In Proceedings of the Workshop on Statistical Machine Translation (WMT), pages 23–30.
    Google ScholarLocate open access versionFindings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS), pages 5998–6008.
    Google ScholarLocate open access versionFindings
  • Mengqiu Wang and Christopher Manning. 2010. Probabilistic tree-edit models with structured latent variables for textual entailment and question answering. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 1164– 1172.
    Google ScholarLocate open access versionFindings
  • Jonathan Weese, Juri Ganitkevitch, and Chris CallisonBurch. 2014. PARADIGM: Paraphrase diagnostics through grammar matching. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL), pages 192–201.
    Google ScholarLocate open access versionFindings
  • Xuchen Yao, Benjamin Van Durme, Chris CallisonBurch, and Peter Clark. 2013. Answer extraction as sequence tagging with tree edit distance. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 858–867.
    Google ScholarLocate open access versionFindings
  • Algorithm B.1 CTED algorithm (Zhang, 1996)
    Google ScholarFindings
  • Da Yin, Tao Meng, and Kai-Wei Chang. 2020. SentiBERT: A transferable transformer-based architecture for compositional sentiment semantics. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 3695– 3706.
    Google ScholarLocate open access versionFindings
  • Kaizhong Zhang. 1996. A constrained edit distance between unordered labeled trees. Algorithmica, 15(3):205–222.
    Google ScholarLocate open access versionFindings
  • Kaizhong Zhang and Tao Jiang. 1994. Some MAX SNP-hard results concerning unordered labeled trees. Information Processing Letters, 49(5):249– 254.
    Google ScholarLocate open access versionFindings
  • Kaizhong Zhang, Jason T. L. Wang, and Dennis Shasha. 1995. On the editing distance between undirected acyclic graphs and related problems. In Annual Symposium on Combinatorial Pattern Matching, pages 395–407.
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科