AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
To address the wrong label problem, we develop innovative solutions that incorporate multi-instance learning into the PCNNS for distant supervised relation extraction

Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks

Conference on Empirical Methods in Natural Language Processing, (2015)

被引用679|浏览242
EI
下载 PDF 全文
引用
微博一下

摘要

Two problems arise when using distant supervision for relation extraction. First, in this method, an already existing knowledge base is heuristically aligned to texts, and the alignment results are treated as labeled data. However, the heuristic alignment can fail, resulting in wrong label problem. In addition, in previous approaches, sta...更多

代码

数据

0
简介
  • One challenge that is faced when building a machine learning system is the generation of training examples.
  • Mentions from free texts matic labeling of data through distant supervision.
  • In this example, Apple and Steve Jobs are two related entities in Freebase.
  • The distant supervision strategy is an effective method of automatically labeling training data.
  • It has two major shortcomings when used for relation extraction
重点内容
  • In relation extraction, one challenge that is faced when building a machine learning system is the generation of training examples
  • We propose a novel model dubbed the Piecewise Convolutional Neural Networks (PCNNs) with multi-instance learning to address these two problems
  • One common technique for coping with this difficulty is distant supervision (Mintz et al, 2009) which assumes that if two entities have a relationship in a known knowledge base, all sentences that mention these two entities will express that relationship in some way
  • To address the wrong label problem, we develop innovative solutions that incorporate multi-instance learning into the PCNNS for distant supervised relation extraction
  • The results show that PCNNs+MIL achieves the best performance; the precision is higher than in the held-out evaluation
  • Features are automatically learned without complicated Natural Language Processing (NLP) preprocessing
方法
  • Distant supervised relation extraction is formulated as multi-instance problem.
  • Figure 3 shows the neural network architecture for distant supervised relation extraction.
  • It illustrates the procedure that handles one instance of a bag
  • This procedure includes four main parts: Vector Representation, Convolution, Piecewise Max Pooling and Softmax Output.
  • The authors describe these parts in detail below
结论
  • The authors exploit Piecewise Convolutional Neural Networks (PCNNs) with multi-instance learning for distant supervised relation extraction.
  • Features are automatically learned without complicated NLP preprocessing.
  • The authors successfully devise a piecewise max pooling layer in the proposed network to capture structural information and incorporate multi-instance learning to address the wrong label problem.
  • Experimental results show that the proposed approach offers significant improvements over comparable methods
总结
  • Introduction:

    One challenge that is faced when building a machine learning system is the generation of training examples.
  • Mentions from free texts matic labeling of data through distant supervision.
  • In this example, Apple and Steve Jobs are two related entities in Freebase.
  • The distant supervision strategy is an effective method of automatically labeling training data.
  • It has two major shortcomings when used for relation extraction
  • Methods:

    Distant supervised relation extraction is formulated as multi-instance problem.
  • Figure 3 shows the neural network architecture for distant supervised relation extraction.
  • It illustrates the procedure that handles one instance of a bag
  • This procedure includes four main parts: Vector Representation, Convolution, Piecewise Max Pooling and Softmax Output.
  • The authors describe these parts in detail below
  • Conclusion:

    The authors exploit Piecewise Convolutional Neural Networks (PCNNs) with multi-instance learning for distant supervised relation extraction.
  • Features are automatically learned without complicated NLP preprocessing.
  • The authors successfully devise a piecewise max pooling layer in the proposed network to capture structural information and incorporate multi-instance learning to address the wrong label problem.
  • Experimental results show that the proposed approach offers significant improvements over comparable methods
表格
  • Table1: Parameters used in our experiments
  • Table2: Precision values for the top 100, top 200, and top 500 extracted relation instances upon manual evaluation
Download tables as Excel
相关工作
  • Relation extraction is one of the most important topics in NLP. Many approaches to relation extraction have been developed, such as bootstrapping, unsupervised relation discovery and supervised classification. Supervised approaches are the most commonly used methods for relation extraction and yield relatively high performance (Bunescu and Mooney, 2006; Zelenko et al, 2003; Zhou et al, 2005). In the supervised paradigm, relation extraction is considered to be a multi-class classification problem and may suffer from a lack of labeled data for training. To address this problem, Mintz et al (2009) adopted Freebase to perform distant supervision. As described in Section 1, the algorithm for training data generation is sometimes faced with the wrong label problem. To address this shortcoming, (Riedel et al, 2010; Hoffmann et al, 2011; Surdeanu et al, 2012) developed the relaxed distant supervision assumption for multi-instance learning. The term ‘multiinstance learning was coined by (Dietterich et al, 1997) while investigating the problem of predicting drug activity. In multi-instance learning, the uncertainty of instance labels can be taken into account. The focus of multi-instance learning is to discriminate among the bags.
基金
  • This work was sponsored by the National Basic Research Program of China (no. 2014CB340503) and the National Natural Science Foundation of China (no. 61272332 and no. 61202329)
引用论文
  • Yoshua Bengio, Rjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155.
    Google ScholarLocate open access versionFindings
  • Razvan C. Bunescu and Raymond J. Mooney. 2005. A shortest path dependency kernel for relation extraction. In Proceedings of HLT/EMNLP, pages 724– 731.
    Google ScholarLocate open access versionFindings
  • Razvan Bunescu and Raymond Mooney. 2006. Subsequence kernels for relation extraction. Proceedings of NIPS, 18:171–178.
    Google ScholarLocate open access versionFindings
  • Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537.
    Google ScholarLocate open access versionFindings
  • Thomas G Dietterich, Richard H Lathrop, and Tomas Lozano-Perez. 1997. Solving the multiple instance problem with axis-parallel rectangles. Journal of Artificial intelligence, 89(1):31–71.
    Google ScholarLocate open access versionFindings
  • Dumitru Erhan, Aaron Courville, Yoshua Bengio, and Pascal Vincent. 2010. Why does unsupervised pretraining help deep learning? Journal of Machine Learning Research, 11:625–660.
    Google ScholarLocate open access versionFindings
  • Benjamin Graham. 2014. Fractional max-pooling. arXiv preprint arXiv:1412.6071.
    Findings
  • Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing coadaptation of feature detectors. arXiv preprint arXiv:1207.0580.
    Findings
  • Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of ACL, pages 541–550.
    Google ScholarLocate open access versionFindings
  • Fei Huang, Arun Ahuja, Doug Downey, Yi Yang, Yuhong Guo, and Alexander Yates. 2014. Learning representations for weakly supervised natural language processing tasks. Journal of Computational Linguistics, 40(1):85–120, March.
    Google ScholarLocate open access versionFindings
  • Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of ACLdemo.
    Google ScholarLocate open access versionFindings
  • Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of EMNLP, pages 1746–1751.
    Google ScholarLocate open access versionFindings
  • Ryan T McDonald and Joakim Nivre. 2007. Characterizing the errors of data-driven dependency parsing models. In Proceedings of EMNLP-CoNLL, pages 122–131.
    Google ScholarLocate open access versionFindings
  • Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of Workshop at ICLR.
    Google ScholarLocate open access versionFindings
  • Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of ACLAFNLP, pages 1003–1011.
    Google ScholarLocate open access versionFindings
  • Ankur P Parikh, Shay B Cohen, and Eric P Xing. 2014. Spectral unsupervised parsing with additive tree metrics. In Proceedings of ACL, pages 1062– 1072.
    Google ScholarLocate open access versionFindings
  • Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP 2014, pages 1746–1751.
    Google ScholarLocate open access versionFindings
  • Longhua Qian, Guodong Zhou, Fang Kong, Qiaoming Zhu, and Peide Qian. 2008. Exploiting constituent dependencies for tree kernel-based semantic relation extraction. In Proceedings of COLING, pages 697– 704.
    Google ScholarLocate open access versionFindings
  • Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Proceedings of ECML PKDD, pages 148–163.
    Google ScholarLocate open access versionFindings
  • Richard Socher, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of EMNLP-CoNLL, pages 1201–1211.
    Google ScholarLocate open access versionFindings
  • Fabian M. Suchanek, Georgiana Ifrim, and Gerhard Weikum. 2006. Combining linguistic and statistical analysis to extract relations from web documents. In Proceedings of KDD, pages 712–717.
    Google ScholarLocate open access versionFindings
  • Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of EMNLP-CoNLL, pages 455–465.
    Google ScholarLocate open access versionFindings
  • Matthew D. Zeiler. 2012. Adadelta: An adaptive learning rate method. arXiv preprint arXiv:1212.5701, abs/1212.5701.
    Findings
  • Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2003. Kernel methods for relation extraction. Journal of Machine Learning Research, 3:1083–1106.
    Google ScholarLocate open access versionFindings
  • Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of COLING, pages 2335–2344.
    Google ScholarLocate open access versionFindings
  • Minling Zhang and Zhihua Zhou. 2006. Adapting rbf neural networks to multi-instance learning. Neural Processing Letters, 23(1):1–26.
    Google ScholarLocate open access versionFindings
  • Min Zhang, Jie Zhang, Jian Su, and Guodong Zhou. 2006. A composite kernel to extract relations between entities with both flat and structured features. In Proceedings of ACL, pages 825–832.
    Google ScholarLocate open access versionFindings
  • Guodong Zhou, Jian Su, Jie Zhang, and Min Zhang. 2005. Exploring various knowledge in relation extraction. In Proceedings of ACL, pages 427–434.
    Google ScholarLocate open access versionFindings
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科