Integrating Local Context and Global Cohesiveness for Open Information Extraction

Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, Volume abs/1804.09931, 2019.

Cited by: 5|Bibtex|Views55|Links
EI
Keywords:
distant supervision entity recognition open information extraction relation extraction weakly-supervised learning
Weibo:
This paper studies the task of open information extraction and proposes a principled framework, ReMine, to unify local contextual information and global structural cohesiveness for effective extraction of relation tuples

Abstract:

Extracting entities and their relations from text is an important task for understanding massive text corpora. Open information extraction (IE) systems mine relation tuples (i.e., entity arguments and a predicate string to describe their relation) from sentences. These relation tuples are not confined to a predefined schema for the relati...More

Code:

Data:

0
Introduction
  • With the emergence of massive text corpora in many domains and languages, the sheer size and rapid growth of this new data poses many challenges understanding and extracting insights from these "! ( ) (!&) () "! ) % ( "#'. )(" ") %## ( ) ( !! ). "!!) & (+,! !"&) " ) ") !! !! %!!
Highlights
  • With the emergence of massive text corpora in many domains and languages, the sheer size and rapid growth of this new data poses many challenges understanding and extracting insights from these "! ( ) (!&) () "! ) % ( "#'

    )(" ") %## ( ) ( !! )

    "!!) & (+,! !"&) " ) ") !! !! %!!
  • In Fig. 6a, we show the distribution of the number of extractions obtained by each Open information extraction system on the first 100 sentences in NYT dataset
  • This paper differs in several aspects: (1) previous work relies on external tools for phrase extraction, which may suffer from domainshift and sparsity problem, while we provide an End-to-End solution towards Open information extraction. (2) previous efforts achieve comparable high precision and reasonable coverage on extraction results, they all focus on local linguistic context
  • This paper studies the task of open information extraction and proposes a principled framework, ReMine, to unify local contextual information and global structural cohesiveness for effective extraction of relation tuples
  • The local objective is jointly learned together with a translating-based objective to enforce structural cohesiveness, such that corpus-level statistics are incorporated for boosting high-quality tuples extracted from individual sentences
  • Experiments on two real-world corpora of different domains demonstrate that ReMine system achieves superior precision when outputting same number of extractions, compared with several state-of-the-art open information extraction systems
Methods
  • For the entity phrase extraction task, NYT and Wiki-KBP are used for evaluation, since both datasets contain annotated entity mentions in test set.
  • ReMine-L by ranking tuples via global cohesiveness without updating entity phrase pairs and any further iterations.
  • In the testing of ReMine and its variants, the authors set hypermargin γ = 1, maximal phrase length ε = 6, number of candidate subject entity phrase for each tail entity Msp = 6 and learning rate of the global cohesiveness module α = 10−3.
  • A more detailed study can be found in Sec. 5
Results
  • The authors use Precision, Recall, and F1-score to evaluate the performances on entity phrase extraction task, same as other sequence labeling studies [24].
  • For the Open IE task, since each tuple obtained by ReMine and other benchmark methods will be assigned a confidence score.
  • MAP is mean average precision of the whole ranking list.
  • Note that the authors do not use recall in this task because it is impossible to collect all the “true" tuples
Conclusion
  • This paper studies the task of open information extraction and proposes a principled framework, ReMine, to unify local contextual information and global structural cohesiveness for effective extraction of relation tuples.
  • The local objective is jointly learned together with a translating-based objective to enforce structural cohesiveness, such that corpus-level statistics are incorporated for boosting high-quality tuples extracted from individual sentences.
  • Experiments on two real-world corpora of different domains demonstrate that ReMine system achieves superior precision when outputting same number of extractions, compared with several state-of-the-art open IE systems.
  • Interesting future work can be (1) On-The-Fly knowledge graph construction from relation tuples; (2) applying
Summary
  • Introduction:

    With the emergence of massive text corpora in many domains and languages, the sheer size and rapid growth of this new data poses many challenges understanding and extracting insights from these "! ( ) (!&) () "! ) % ( "#'. )(" ") %## ( ) ( !! ). "!!) & (+,! !"&) " ) ") !! !! %!!
  • Methods:

    For the entity phrase extraction task, NYT and Wiki-KBP are used for evaluation, since both datasets contain annotated entity mentions in test set.
  • ReMine-L by ranking tuples via global cohesiveness without updating entity phrase pairs and any further iterations.
  • In the testing of ReMine and its variants, the authors set hypermargin γ = 1, maximal phrase length ε = 6, number of candidate subject entity phrase for each tail entity Msp = 6 and learning rate of the global cohesiveness module α = 10−3.
  • A more detailed study can be found in Sec. 5
  • Results:

    The authors use Precision, Recall, and F1-score to evaluate the performances on entity phrase extraction task, same as other sequence labeling studies [24].
  • For the Open IE task, since each tuple obtained by ReMine and other benchmark methods will be assigned a confidence score.
  • MAP is mean average precision of the whole ranking list.
  • Note that the authors do not use recall in this task because it is impossible to collect all the “true" tuples
  • Conclusion:

    This paper studies the task of open information extraction and proposes a principled framework, ReMine, to unify local contextual information and global structural cohesiveness for effective extraction of relation tuples.
  • The local objective is jointly learned together with a translating-based objective to enforce structural cohesiveness, such that corpus-level statistics are incorporated for boosting high-quality tuples extracted from individual sentences.
  • Experiments on two real-world corpora of different domains demonstrate that ReMine system achieves superior precision when outputting same number of extractions, compared with several state-of-the-art open IE systems.
  • Interesting future work can be (1) On-The-Fly knowledge graph construction from relation tuples; (2) applying
Tables
  • Table1: Entity and relation phrase candidates generation with regular expression patterns on part-of-speech tag
  • Table2: Features used in the phrase extraction module (Sec. 3.1)
  • Table3: Performance comparison with state-of-the-art entity phrase extraction algorithms for the weakly-supervised entity phrase extraction task
  • Table4: Performance comparison with state-of-the-art Open IE systems on two datasets from different domains, using Precision@K, Mean Average Precision (MAP), Normalized Discounted Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR)
  • Table5: Extraction samples of one sentence in the NYT dataset using different methods. “T” means correct tuples and “F” means incorrect ones. ∗The tuple is too complicated to clearly explain one proposition. #The tuple cannot read smoothly. †The tuple is logically wrong
  • Table6: Different entity pairs discovered by ReMine and ReMine-G, where blue ones are incorrect extractions
Download tables as Excel
Related work
  • Open Information Extraction. Open domain information extraction has been extensively studied in literature. Most of the existing work follow two lines of work, that is, pattern based methods or clause based methods. Pattern based information extraction can be as early as Hearst patterns like “N P0 such as {N P1, N P2, ...}” for hyponymy relation extraction [16]. Carlson and Mitchell et al.

    introduced Never-Ending Language Learning (NELL) based on freetext predicate patterns [7, 26]. ReVerb [12] identified relational phrases via part-of-speech-based regular expressions. Besides partof-speech tags, recent works have started to use more linguistic features, such as dependency parsing, to induce long distance relationships [27, 32]. Similarly, ClausIE [10] inducted short but coherent pieces of information along dependency paths, which is typically subject, predicate and optional object with complement. Angeli et al adopts a clause splitter using distant training and statistically maps predicate to known relation schemas [2]. MinIE [14] removes overly-specific constituents and captures implicit relations in ClausIE by introducing several statistical measures like polarity, modality, attribution, and quantities. Compared with these works, this paper differs in several aspects: (1) previous work relies on external tools for phrase extraction, which may suffer from domainshift and sparsity problem, while we provide an End-to-End solution towards Open IE. (2) Although previous efforts achieve comparable high precision and reasonable coverage on extraction results, they all focus on local linguistic context. The correctness of extracted facts are evaluated purely on local context, however, large corpus can exclude false extractions from inferred inconsistencies.
Funding
  • Research was sponsored in part by the U.S Army Research Lab. under Cooperative Agreement No W911NF-09-2-0053 (NSCTA), National Science Foundation IIS 16-18481, IIS 17-04532, and IIS-1741317, and grant 1U54GM114838 awarded by NIGMS through funds provided by the trans-NIH Big Data to Knowledge (BD2K) initiative (www.bd2k.nih.gov)
  • Xiang Ren’s research has been supported in part by National Science Foundation SMA 18-29268
Reference
  • Armen Allahverdyan and Aram Galstyan. 201Comparative analysis of viterbi training and maximum likelihood estimation for hmms. In NIPS.
    Google ScholarFindings
  • Gabor Angeli, Melvin Johnson Premkumar, and Christopher D Manning. 2015. Leveraging linguistic structure for open domain information extraction. In ACL.
    Google ScholarFindings
  • Michele Banko, Michael J Cafarella, Stephen Soderland, Matt Broadhead, and Oren Etzioni. 2007. Open information extraction from the web. In IJCAI.
    Google ScholarLocate open access versionFindings
  • Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD.
    Google ScholarFindings
  • Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS.
    Google ScholarFindings
  • Razvan C. Bunescu and Raymond J. Mooney. 2007. Learning to Extract Relations from the Web using Minimal Supervision. In ACL.
    Google ScholarFindings
  • Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka Jr, and Tom M Mitchell. 2010. Toward an architecture for never-ending language learning. In AAAI.
    Google ScholarFindings
  • Andrew Carlson, Justin Betteridge, Richard C Wang, Estevam R Hruschka Jr, and Tom M Mitchell. 2010. Coupled semi-supervised learning for information extraction. In WSDM.
    Google ScholarFindings
  • Joachim Daiber, Max Jakob, Chris Hokamp, and Pablo N. Mendes. 2013. Improving Efficiency and Accuracy in Multilingual Entity Extraction. In I-Semantics.
    Google ScholarFindings
  • Luciano Del Corro and Rainer Gemulla. 2013. Clausie: clause-based open information extraction. In WWW.
    Google ScholarFindings
  • Xin Luna Dong, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge Vault: A Web-scale approach to probabilistic knowledge fusion. In KDD.
    Google ScholarFindings
  • Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In EMNLP.
    Google ScholarFindings
  • Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2014. Open Question Answering Over Curated and Extracted Knowledge Bases. KDD (2014).
    Google ScholarLocate open access versionFindings
  • Kiril Gashteovski, Rainer Gemulla, and Luciano Del Corro. 2017. MinIE: minimizing facts in open information extraction. In EMNLP.
    Google ScholarFindings
  • Kelvin Guu, John Miller, and Percy Liang. 20Traversing knowledge graphs in vector space. In EMNLP.
    Google ScholarFindings
  • Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In ACL.
    Google ScholarFindings
  • Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al. 2015. DBpedia–a large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web 6, 2 (2015), 167–195.
    Google ScholarLocate open access versionFindings
  • Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion.. In AAAI, Vol. 15. 2181–2187.
    Google ScholarLocate open access versionFindings
  • Xiao Ling and Daniel S Weld. 2012. Fine-Grained Entity Recognition.. In AAAI.
    Google ScholarLocate open access versionFindings
  • Jialu Liu, Jingbo Shang, and Jiawei Han. [n. d.]. Phrase Mining from Massive Text and Its Applications. Vol. 9.
    Google ScholarFindings
  • Jialu Liu, Jingbo Shang, Chi Wang, Xiang Ren, and Jiawei Han. 2015. Mining quality phrases from massive text corpora. In SIGMOD.
    Google ScholarFindings
  • Liyuan Liu, Jingbo Shang, Frank F. Xu, Xiang Ren, Huan Gui, Jian Peng, and Jiawei Han. 2017. Empower sequence labeling with task-aware neural language model. arXiv preprint arXiv:1709.04109 (2017).
    Findings
  • Yuanfei Luo, Quan Wang, Bin Wang, and Li Guo. 2015. Context-dependent knowledge graph embedding.. In EMNLP.
    Google ScholarLocate open access versionFindings
  • Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bidirectional lstm-cnns-crf. In ACL.
    Google ScholarFindings
  • Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The stanford corenlp natural language processing toolkit.. In ACL (System Demonstrations).
    Google ScholarLocate open access versionFindings
  • Tom M Mitchell, William W Cohen, Estevam R Hruschka Jr, Partha Pratim Talukdar, Justin Betteridge, Andrew Carlson, Bhavana Dalvi Mishra, Matthew Gardner, Bryan Kisiel, Jayant Krishnamurthy, et al. 2015. Never ending learning. In AAAI.
    Google ScholarFindings
  • Ndapandula Nakashole, Gerhard Weikum, and Fabian Suchanek. 2012. PATTY: A taxonomy of relational patterns with semantic types. In EMNLP-CoNLL.
    Google ScholarFindings
  • Feng Niu, Ce Zhang, Christopher Ré, and Jude W Shavlik. 2012. Deepdive: Webscale knowledge-base construction using statistical learning and inference.. In VLDS.
    Google ScholarFindings
  • Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In NAACL.
    Google ScholarFindings
  • Tim Rocktäschel, Sameer Singh, and Sebastian Riedel. 2015. Injecting logical background knowledge into embeddings for relation extraction. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1119–1129.
    Google ScholarLocate open access versionFindings
  • Sunita Sarawagi. 2008. Information extraction. Foundations and trends in databases 1, 3 (2008), 261–377.
    Google ScholarFindings
  • Michael Schmitz, Robert Bart, Stephen Soderland, Oren Etzioni, et al. 2012. Open language learning for information extraction. In EMNLP-CoNLL.
    Google ScholarFindings
  • Jingbo Shang, Jialu Liu, Meng Jiang, Xiang Ren, Clare R Voss, and Jiawei Han. 2017. Automated phrase mining from massive text corpora. arXiv preprint arXiv:1702.04457 (2017).
    Findings
  • Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Advances in neural information processing systems. 926–934.
    Google ScholarLocate open access versionFindings
  • Huan Sun, Hao Ma, Wen-tau Yih, Chen-Tse Tsai, Jingjing Liu, and Ming-Wei Chang. 2015. Open domain question answering via semantic enrichment. In WWW.
    Google ScholarFindings
  • Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1499–1509.
    Google ScholarLocate open access versionFindings
  • Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes.. In AAAI.
    Google ScholarFindings
  • Chao Zhang, Guangyu Zhou, Quan Yuan, Honglei Zhuang, Yu Zheng, Lance Kaplan, Shaowen Wang, and Jiawei Han. 2016. Geoburst: Real-time local event detection in geo-tagged tweet streams. In SIGIR.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments