AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
We achieve state-of-the-art results on multiple benchmark knowledge base completion tasks and we show that our model is robust and can learn long chains-ofreasoning

Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning.

international conference on learning representations, (2018)

被引用185|浏览643
EI
下载 PDF 全文
引用
微博一下

摘要

Knowledge bases (KB), both automatically and manually constructed, are often incomplete --- many valid facts can be inferred from the KB by synthesizing existing information. A popular approach to KB completion is to infer new relations by combinatory reasoning over the information found along other paths connecting a pair of entities. Gi...更多

代码

数据

0
简介
  • The ability of computing systems to make new inferences from observed evidence, has been a long standing goal of artificial intelligence.
  • The authors are interested in automated reasoning on large knowledge bases (KB) with rich and diverse semantics (Suchanek et al, 2007; Bollacker et al, 2008; Carlson et al, 2010).
  • The authors' goal is to automatically learn such reasoning paths in KBs. The authors frame the learning problem as one of query answering, that is to say, answering questions of the form (Colin Kaepernick, PlaysInLeague, ?)
重点内容
  • Automated reasoning, the ability of computing systems to make new inferences from observed evidence, has been a long standing goal of artificial intelligence
  • This paper presents a method for efficiently searching the graph for answer-providing paths using reinforcement learning (RL) conditioned on the input question, eliminating any need for precomputed paths
  • The main contributions of the paper are: (a) We present agent MINERVA, which learns to do query answering by walking on a knowledge graph conditioned on an input query, stopping when it reaches the answer node
  • The agent is trained using reinforcement learning, policy gradients (§ 2). (b) We evaluate MINERVA on several benchmark datasets and compare favorably to Neural Theorem Provers (NTP) (Rocktaschel & Riedel, 2017) and Neural LP (Yang et al, 2017), which do logical rule learning in knowledge bases (KB), and state-of-the-art embedding based methods such as DistMult (Yang et al, 2015) and ComplEx (Trouillon et al, 2016). (c) We extend MINERVA to handle partially structured natural language queries and test it on the WikiMovies dataset (§ 4.3) (Miller et al, 2016)
  • We explored a new way of automated reasoning on large knowledge bases in which we use the knowledge graphs representation of the knowledge base and train an agent to walk to the answer node conditioned on the input query
  • We achieve state-of-the-art results on multiple benchmark knowledge base completion tasks and we show that our model is robust and can learn long chains-ofreasoning
方法
  • 4.1 KNOWLEDGE GRAPH QUERY ANSWERING

    This section describes the experimental results on the various knowledge graph query answering datasets.
  • COUNTRIES dataset which is explicitly designed to test the ability NeuralLP 0.70 0.73 of models to learn logical rules.
  • It contains countries, regions and MINERVA 0.91 0.93 subregions as entities.
  • NTP-λ is a NTP model trained with an additional objective function of ComplEx. The authors compare MINERVA against Neural LP (Yang et al, 2017) on the UMLS and KINSHIP datasets
结果
  • The authors achieve state-of-the-art results on multiple benchmark knowledge base completion tasks and the authors show that the model is robust and can learn long chains-ofreasoning.
结论
  • The authors explored a new way of automated reasoning on large knowledge bases in which the authors use the knowledge graphs representation of the knowledge base and train an agent to walk to the answer node conditioned on the input query.
  • The authors achieve state-of-the-art results on multiple benchmark knowledge base completion tasks and the authors show that the model is robust and can learn long chains-ofreasoning.
  • It needs no pretraining or initial supervision.
  • Future research directions include applying more sophisticated RL techniques and working directly on textual queries and documents
总结
  • Introduction:

    The ability of computing systems to make new inferences from observed evidence, has been a long standing goal of artificial intelligence.
  • The authors are interested in automated reasoning on large knowledge bases (KB) with rich and diverse semantics (Suchanek et al, 2007; Bollacker et al, 2008; Carlson et al, 2010).
  • The authors' goal is to automatically learn such reasoning paths in KBs. The authors frame the learning problem as one of query answering, that is to say, answering questions of the form (Colin Kaepernick, PlaysInLeague, ?)
  • Objectives:

    The authors can infer the home stadium of Colin Kaepernick from the following reasoning path: Colin Kaepernick → PlaysInTeam → 49ers → TeamHomeStadium → Levi’s Stadium.
  • Methods:

    4.1 KNOWLEDGE GRAPH QUERY ANSWERING

    This section describes the experimental results on the various knowledge graph query answering datasets.
  • COUNTRIES dataset which is explicitly designed to test the ability NeuralLP 0.70 0.73 of models to learn logical rules.
  • It contains countries, regions and MINERVA 0.91 0.93 subregions as entities.
  • NTP-λ is a NTP model trained with an additional objective function of ComplEx. The authors compare MINERVA against Neural LP (Yang et al, 2017) on the UMLS and KINSHIP datasets
  • Results:

    The authors achieve state-of-the-art results on multiple benchmark knowledge base completion tasks and the authors show that the model is robust and can learn long chains-ofreasoning.
  • Conclusion:

    The authors explored a new way of automated reasoning on large knowledge bases in which the authors use the knowledge graphs representation of the knowledge base and train an agent to walk to the answer node conditioned on the input query.
  • The authors achieve state-of-the-art results on multiple benchmark knowledge base completion tasks and the authors show that the model is robust and can learn long chains-ofreasoning.
  • It needs no pretraining or initial supervision.
  • Future research directions include applying more sophisticated RL techniques and working directly on textual queries and documents
表格
  • Table1: Statistics of various datasets used in experiments
  • Table2: Performance on COUNTRIES dataset. MINERVA significantly outperforms baselines in the challenging S3 task
  • Table3: HITS@10 on UMLS and
  • Table4: MAP scores for different query relations on the NELL-995 dataset. Note that in this comparison, MINERVA refers to only a single learnt model for all query relations which is competitive with individual DeepPath models trained separately for each query relation
  • Table5: Performance on WN18RR
  • Table6: Performance on FB15K-237
  • Table7: Performance on WikiMovies written by Herb Freed?”. WikiMovies also has an accompanying KB which can be used to answer all the questions
  • Table8: A few example of paths found by MINERVA on the COUNTRIES and NELL. MINERVA can learn general rules as required by the COUNTRIES dataset (example (i)). It can learn shorter paths if necessary (example (ii)) and has the ability to correct a previously taken decision (example (iii))
  • Table9: Few example facts belonging to m to 1, 1 to m relations in FB15k-237
  • Table10: Few example 1-to-M relations from FB15k-237 with high cardinality ratio of tail to head
  • Table11: Best hyper parameters
Download tables as Excel
相关工作
  • Learning vector representations of entities and relations using tensor factorization (Nickel et al, 2011; 2012; Bordes et al, 2013; Riedel et al, 2013; Nickel et al, 2014; Yang et al, 2015) or neural methods (Socher et al, 2013; Toutanova et al, 2015; Verga et al, 2016) has been a popular approach to reasoning with a knowledge base. However, these methods cannot capture more complex reasoning patterns such as those found by following inference paths in KBs. Multi-hop link prediction approaches (Lao et al, 2011; Neelakantan et al, 2015; Guu et al, 2015; Toutanova et al, 2016; Das et al, 2017) address the problems above, but the reasoning paths that they operate on are gathered by (i) Can learn general rules:

    (S1) LocatedIn(X, Y) ← LocatedIn(X, Z) & LocatedIn(Z, Y) (S2) LocatedIn(X, Y) ← NeighborOf(X, Z) & LocatedIn(Z, Y) (S3) LocatedIn(X, Y) ← NeighborOf(X, Z) & NeighborOf(Z, W) & LocatedIn(W, Y)

    (ii) Can learn shorter path: Richard F. Velky −W−o−rk−s−Fo→r ?

    Richard F. Velky −P−e−rs−on−L−ea−d−sO−r→g Schaghticokes −N−O−-−O→P Schaghticokes −N−O−-−O→P Schaghticokes (iii) Can recover from mistakes: Donald Graham −W−o−rk−s−Fo→r ?

    Donald Graham −O−r−gT−e−rm−i−na−te−d−Pe−r−so→n TNT Post −O−rg−T−er−m−in−a−te−dP−e−rs−o−n−→1 Donald Graham −O−r−gH−i−re−dP−e−rs−o→n Wash Post performing random walks independent of the type of query relation. Lao et al (2011) further filters paths from the set of sampled paths based on the restriction that the path must end at one of the target entities in the training set and are within a maximum length. These constraints make them query dependent but they are heuristic in nature. Our approach eliminates any necessity to pre-compute paths and learns to efficiently search the graph conditioned on the input query relation.
基金
  • This work was supported in part by the Center for Data Science and the Center for Intelligent Information Retrieval, in part by DARPA under agreement number FA8750-13-2-0020, in part by Defense Advanced Research Agency (DARPA) contract number HR0011-15-2-0036, in part by the National Science Foundation (NSF) grant numbers DMR-1534431 and IIS-1514053 and in part by the Chan Zuckerberg Initiative under the project Scientific Knowledge Base Construction
引用论文
  • Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: A collaboratively created graph database for structuring human knowledge. In ICDM, 2008.
    Google ScholarLocate open access versionFindings
  • Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, 2013.
    Google ScholarFindings
  • Guillaume Bouchard, Sameer Singh, and Theo Trouillon. On approximate reasoning capabilities of low-rank vector spaces. AAAI Spring Symposium, 2015.
    Google ScholarLocate open access versionFindings
  • Matthias Broecheler, Lilyana Mihalkova, and Lise Getoor. Probabilistic similarity logic. In UAI, 2010.
    Google ScholarLocate open access versionFindings
  • Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka, Jr., and Tom M. Mitchell. Toward an Architecture for Never-ending Language Learning. In AAAI, 2010.
    Google ScholarLocate open access versionFindings
  • Kai-Wei Chang, Akshay Krishnamurthy, Alekh Agarwal, Hal Daume, and John Langford. Learning to search better than your teacher. In ICML, 2015.
    Google ScholarLocate open access versionFindings
  • William Cohen. Tensorlog: A differentiable deductive database. arXiv:1605.06523, 2016.
    Findings
  • Michael Collins and Brian Roark. Incremental parsing with the perceptron algorithm. In ACL, 2004.
    Google ScholarLocate open access versionFindings
  • Rajarshi Das, Arvind Neelakantan, David Belanger, and Andrew McCallum. Chains of reasoning over entities, relations, and text using recurrent neural networks. In EACL, 2017.
    Google ScholarLocate open access versionFindings
  • Hal Daume III and Daniel Marcu. Learning as search optimization: Approximate large margin methods for structured prediction. In ICML, 2005.
    Google ScholarLocate open access versionFindings
  • Hal Daume III, John Langford, and Daniel Marcu. Search-based structured prediction. Machine learning, 2009.
    Google ScholarFindings
  • Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. arXiv:1707.01476, 2017.
    Findings
  • Michael Evans and Timothy Swartz. Approximating integrals via Monte Carlo and deterministic methods. OUP Oxford, 2000.
    Google ScholarFindings
  • George Fishman. Monte Carlo: concepts, algorithms, and applications. Springer Science & Business Media, 2013.
    Google ScholarFindings
  • Lise Getoor and Ben Taskar. Introduction to statistical relational learning. MIT press, 2007.
    Google ScholarFindings
  • Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv:1410.5401, 2014.
    Findings
  • Kelvin Guu, John Miller, and Percy Liang. Traversing knowledge graphs in vector space. In EMNLP, 2015.
    Google ScholarLocate open access versionFindings
  • John Hammersley. Monte carlo methods. Springer Science & Business Media, 2013.
    Google ScholarFindings
  • Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. Neural computation, 1997.
    Google ScholarLocate open access versionFindings
  • Paul W Holland and Samuel Leinhardt. Transitivity in structural models of small groups. Comparative Group Studies, 1971.
    Google ScholarLocate open access versionFindings
  • Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014.
    Findings
  • Stanley Kok and Pedro Domingos. Statistical predicate invention. In ICML, 2007.
    Google ScholarLocate open access versionFindings
  • Ni Lao, Tom Mitchell, and William Cohen. Random walk inference and learning in a large scale knowledge base. In EMNLP, 2011.
    Google ScholarLocate open access versionFindings
  • John McCarthy. Programs with common sense. RLE and MIT Computation Center, 1960.
    Google ScholarLocate open access versionFindings
  • Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. Key-value memory networks for directly reading documents. EMNLP, 2016.
    Google ScholarLocate open access versionFindings
  • Bonan Min, Ralph Grishman, Li Wan, Chang Wang, and David Gondek. Distant supervision for relation extraction with an incomplete knowledge base. In HLT-NAACL, 2013.
    Google ScholarFindings
  • Stephen Muggleton. Inverse entailment and progol. New generation computing, 1995.
    Google ScholarFindings
  • Stephen Muggleton, Ramon Otero, and Alireza Tamaddoni-Nezhad. Inductive logic programming. Springer, 1992.
    Google ScholarFindings
  • Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. Compositional vector space models for knowledge base completion. In ACL, 2015.
    Google ScholarLocate open access versionFindings
  • Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A three-way model for collective learning on multi-relational data. In ICML, 2011.
    Google ScholarLocate open access versionFindings
  • Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. Factorizing yago: scalable machine learning for linked data. In WWW, 2012.
    Google ScholarLocate open access versionFindings
  • Maximilian Nickel, Xueyan Jiang, and Volker Tresp. Reducing the rank in relational factorization models by including observable patterns. In NIPS, 2014.
    Google ScholarLocate open access versionFindings
  • Nils J Nilsson. Logic and artificial intelligence. Artificial intelligence, 1991.
    Google ScholarLocate open access versionFindings
  • J Ross Quinlan. Learning logical definitions from relations. Machine learning, 1990.
    Google ScholarFindings
  • Matthew Richardson and Pedro Domingos. Markov logic networks. Machine learning, 2006.
    Google ScholarFindings
  • Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. Relation extraction with matrix factorization and universal schemas. In NAACL, 2013.
    Google ScholarFindings
  • Tim Rocktaschel and Sebastian Riedel. End-to-end differentiable proving. In NIPS, 2017.
    Google ScholarLocate open access versionFindings
  • Stephane Ross, Geoffrey J Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In AISTATS, 2011.
    Google ScholarLocate open access versionFindings
  • Stefan Schoenmackers, Oren Etzioni, Daniel Weld, and Jesse Davis. Learning first-order horn clauses from web text. In EMNLP, 2010.
    Google ScholarLocate open access versionFindings
  • Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. Reasonet: Learning to stop reading in machine comprehension. In KDD, 2017.
    Google ScholarLocate open access versionFindings
  • Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. Reasoning with neural tensor networks for knowledge base completion. In NIPS, 2013.
    Google ScholarFindings
  • Fabian Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: A core of semantic knowledge. In WWW, 2007.
    Google ScholarFindings
  • Sainbayar Sukhbaatar, Jason Weston, and Rob Fergus. End-to-end memory networks. In NIPS, 2015.
    Google ScholarLocate open access versionFindings
  • Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In EMNLP, 2015.
    Google ScholarLocate open access versionFindings
  • Kristina Toutanova, Victoria Lin, Wen-tau Yih, Hoifung Poon, and Chris Quirk. Compositional learning of embeddings for relation paths in knowledge base and text. In ACL, 2016.
    Google ScholarLocate open access versionFindings
  • Theo Trouillon, Johannes Welbl, Sebastian Riedel, Eric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In ICML, 2016.
    Google ScholarLocate open access versionFindings
  • Patrick Verga, David Belanger, Emma Strubell, Benjamin Roth, and Andrew McCallum. Multilingual relation extraction using compositional universal schema. In NAACL, 2016.
    Google ScholarLocate open access versionFindings
  • William Yang Wang, Kathryn Mazaitis, and William W Cohen. Programming with personalized pagerank: a locally groundable first-order probabilistic logic. In CIKM, 2013.
    Google ScholarLocate open access versionFindings
  • Duncan J Watts and Steven H Strogatz. Collective dynamics of small-worldnetworks. nature, 1998.
    Google ScholarLocate open access versionFindings
  • Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 1992.
    Google ScholarFindings
  • Wenhan Xiong, Thien Hoang, and William Yang Wang. Deeppath: A reinforcement learning method for knowledge graph reasoning. In EMNLP, 2017.
    Google ScholarLocate open access versionFindings
  • Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In ICLR, 2015.
    Google ScholarLocate open access versionFindings
  • Fan Yang, Zhilin Yang, and William W Cohen. Differentiable learning of logical rules for knowledge base reasoning. In NIPS, 2017.
    Google ScholarLocate open access versionFindings
  • Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. Deep sets. In NIPS, 2017.
    Google ScholarLocate open access versionFindings
  • Wojciech Zaremba and Ilya Sutskever. Reinforcement learning neural turing machines. arXiv:1505.00521, 2015.
    Findings
您的评分 :
0

 

标签
评论
小科