Neural Cross-Lingual Coreference Resolution And Its Application To Entity Linking

ACL, pp. 395-400, 2018.

Cited by: 13|Views45
EI
Weibo:
A coreference model trained on English data is unlikely to coreference these two mentions in Spanish since these mentions did not appear in English data and a regular English style abbreviation of “Estados Unidos” will be “EU” instead of “EEUU”

Abstract:

We propose an entity-centric neural cross-lingual coreference model that builds on multi-lingual embeddings and language-independent features. We perform both intrinsic and extrinsic evaluations of our model. In the intrinsic evaluation, we show that our model, when trained on English and tested on Chinese and Spanish, achieves competitiv...More

Code:

Data:

0
Full Text
Bibtex
Weibo
Introduction
  • Cross-lingual models for NLP tasks are important since they can be used on data from a new language without requiring annotation from the new language (Ji et al, 2014, 2015).
  • The mentions “EEUU” (“US” in English) and “Estados Unidos” (“United States” in English) are coreferent.
  • In the bilingual EnglishSpanish word embedding space, the word embedding of “EEUU” sits close to the word embedding of “US” and the sum of word embeddings of “Estados Unidos” sit close to the sum of word embeddings of “United States”.
  • A coreference model trained using English-Spanish bilingual word embeddings on English data has the potential to make the correct coreference decision between “EEUU” and “Estados Unidos” without ever encountering these mentions in training data
Highlights
  • Cross-lingual models for NLP tasks are important since they can be used on data from a new language without requiring annotation from the new language (Ji et al, 2014, 2015)
  • This paper investigates the use of multi-lingual embeddings (Faruqui and Dyer, 2014; Upadhyay et al, 2016) for building cross-lingual models for the task of coreference resolution (Ng and Cardie, 2002; Pradhan et al, 2012)
  • Unos 100 millones de personas enfrentaban el sabado nuevas dificultades tras la enorme tormenta de nieve de hace dıas en la costa este de Estados Unidos.”
  • A coreference model trained on English data is unlikely to coreference these two mentions in Spanish since these mentions did not appear in English data and a regular English style abbreviation of “Estados Unidos” will be “EU” instead of “EEUU”
  • In this work, we focus on coreference on mentions that arise in our end task of entity linking and conduct experiments on TAC TriLingual 2015 data sets consisting of English, Chinese and Spanish
  • The model achieves state-of-the-art (SOTA) results on English benchmark EL datasets and performs surprisingly well on Spanish and Chinese
  • We evaluate cross-lingual transfer of coreference models on the TAC 2015 Tri-Lingual EL datasets
Methods
  • The authors evaluate cross-lingual transfer of coreference models on the TAC 2015 Tri-Lingual EL datasets.
  • It contains mentions annotated with their grounded Freebase 1 links or corpus-wide clustering information for 3 languages: English, Chinese and Spanish.
  • The documents come from two genres of newswire and discussion forums
  • The mentions in this dataset are either named entities or nominals that belong to five types: PER, ORG, GPE, LOC and FAC.
  • The authors use SGD for optimization with an initial learning rate of 0.05 which is linearly reduced to
Results
  • The model achieves state-of-the-art (SOTA) results on English benchmark EL datasets and performs surprisingly well on Spanish and Chinese.
Conclusion
  • The proposed cross-lingual coreference model was found to be empirically strong in both intrinsic and extrinsic evaluations in the context of an entity linking task.
Summary
  • Introduction:

    Cross-lingual models for NLP tasks are important since they can be used on data from a new language without requiring annotation from the new language (Ji et al, 2014, 2015).
  • The mentions “EEUU” (“US” in English) and “Estados Unidos” (“United States” in English) are coreferent.
  • In the bilingual EnglishSpanish word embedding space, the word embedding of “EEUU” sits close to the word embedding of “US” and the sum of word embeddings of “Estados Unidos” sit close to the sum of word embeddings of “United States”.
  • A coreference model trained using English-Spanish bilingual word embeddings on English data has the potential to make the correct coreference decision between “EEUU” and “Estados Unidos” without ever encountering these mentions in training data
  • Objectives:

    The authors' aim is to apply the proposed coreference model to the EL system to perform an extrinsic evaluation of the proposed algorithm.
  • Methods:

    The authors evaluate cross-lingual transfer of coreference models on the TAC 2015 Tri-Lingual EL datasets.
  • It contains mentions annotated with their grounded Freebase 1 links or corpus-wide clustering information for 3 languages: English, Chinese and Spanish.
  • The documents come from two genres of newswire and discussion forums
  • The mentions in this dataset are either named entities or nominals that belong to five types: PER, ORG, GPE, LOC and FAC.
  • The authors use SGD for optimization with an initial learning rate of 0.05 which is linearly reduced to
  • Results:

    The model achieves state-of-the-art (SOTA) results on English benchmark EL datasets and performs surprisingly well on Spanish and Chinese.
  • Conclusion:

    The proposed cross-lingual coreference model was found to be empirically strong in both intrinsic and extrinsic evaluations in the context of an entity linking task.
Tables
  • Table1: No of documents for the TAC 2015 TriLingual EL Dataset
  • Table2: Coreference results on the En test set of TAC 15 competition. Our model significantly outperforms C&M16
  • Table3: Coreference results on the Es and Zh test sets of TAC 15. En model performs competitively to the models trained on target language data
  • Table4: Performance comparison on the TAC 2015 Es and Zh datasets. EL + En Coref outperforms the best 2015 TAC system (Rank 1) without requiring any Es or Zh coreference data
Download tables as Excel
Related work
Reference
  • Waleed Ammar, George Mulcaire, Yulia Tsvetkov, Guillaume Lample, Chris Dyer, and Noah A Smith. 2016. Massively multilingual word embeddings. arXiv preprint arXiv:1602.01925.
    Findings
  • Eric Bengtson and Dan Roth. 2008. Understanding the value of features for coreference resolution. In EMNLP.
    Google ScholarFindings
  • Anders Bjorkelund and Jonas Kuhn. 2014. Learning structured perceptrons for coreference resolution with latent antecedents and non-local features. In ACL.
    Google ScholarFindings
  • Kevin Clark and Christopher D Manning. 2015. Entitycentric coreference resolution with model stacking. In ACL.
    Google ScholarFindings
  • Kevin Clark and Christopher D Manning. 2016. Improving coreference resolution by learning entitylevel distributed representations. In ACL.
    Google ScholarFindings
  • Greg Durrett, David Leo Wright Hall, and Dan Klein. 2013. Decentralized entity-level modeling for coreference resolution. In ACL.
    Google ScholarFindings
  • Greg Durrett and Dan Klein. 2014. A joint model for entity analysis: Coreference, typing, and linking. Transactions of the Association for Computational Linguistics, 2.
    Google ScholarLocate open access versionFindings
  • Manaal Faruqui and Chris Dyer. 2014. Improving vector space word representations using multilingual correlation. In EACL.
    Google ScholarFindings
  • Eraldo Rezende Fernandes, Cıcero Nogueira Dos Santos, and Ruy Luiz Milidiu. 2012. Latent structure perceptron with feature induction for unrestricted coreference resolution. In EMNLP-CoNLL.
    Google ScholarFindings
  • Hannaneh Hajishirzi, Leila Zilles, Daniel S Weld, and Luke Zettlemoyer. 2013. Joint coreference resolution and named-entity linking with multi-pass sieves. In EMNLP.
    Google ScholarFindings
  • Heng Ji, Joel Nothman, Ben Hachey, and Radu Florian. 2015. Overview of tac-kbp2015 tri-lingual entity discovery and linking. In TAC.
    Google ScholarFindings
  • Heng Ji, Joel Nothman, Ben Hachey, et al. 2014. Overview of tac-kbp2014 entity discovery and linking tasks. In TAC.
    Google ScholarFindings
  • Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. arXiv preprint arXiv:1707.07045.
    Findings
  • Sebastian Martschat and Michael Strube. 2015. Latent structures for coreference resolution. Transactions of the Association for Computational Linguistics, 3:405–418.
    Google ScholarLocate open access versionFindings
  • Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
    Findings
  • Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013b. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168.
    Findings
  • Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In ACL.
    Google ScholarFindings
  • Jian Ni, Georgiana Dinu, and Radu Florian. 2017. Weakly supervised cross-lingual named entity recognition via effective annotation and representation projection. In ACL.
    Google ScholarFindings
  • Mark Palatucci, Dean Pomerleau, Geoffrey E Hinton, and Tom M Mitchell. 2009. Zero-shot learning with semantic output codes. In NIPS.
    Google ScholarFindings
  • Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In EMNLPCoNLL.
    Google ScholarFindings
  • Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In CoNLL.
    Google ScholarFindings
  • Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nathanael Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher Manning. 2010. A multipass sieve for coreference resolution. In EMNLP.
    Google ScholarFindings
  • Altaf Rahman and Vincent Ng. 2009. Supervised models for coreference resolution. In EMNLP.
    Google ScholarFindings
  • Avirup Sil, Georgiana Dinu, and Radu Florian. 2015. The ibm systems for trilingual entity discovery and linking at tac 2015. In TAC.
    Google ScholarFindings
  • Avirup Sil and Radu Florian. 2016. One for all: Towards language independent named entity linking. In ACL.
    Google ScholarFindings
  • Avirup Sil, Gourab Kundu, Radu Florian, and Wael Hamza. 2018. Neural cross-lingual entity linking. In AAAI.
    Google ScholarLocate open access versionFindings
  • Richard Socher, Milind Ganjoo, Christopher D Manning, and Andrew Ng. 2013. Zero-shot learning through cross-modal transfer. In NIPS.
    Google ScholarFindings
  • Chen-Tse Tsai and Dan Roth. 2016. Cross-lingual wikification using multilingual embeddings. In HLTNAACL.
    Google ScholarFindings
  • Shyam Upadhyay, Manaal Faruqui, Chris Dyer, and Dan Roth. 2016. Cross-lingual models of word embeddings: An empirical comparison. In ACL.
    Google ScholarFindings
  • Sam Wiseman, Alexander M Rush, and Stuart M Shieber. 2016. Learning global features for coreference resolution. In NAACL.
    Google ScholarFindings
  • Sam Joshua Wiseman, Alexander Matthew Rush, Stuart Merrill Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In ACL.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments