AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
The results of extensive experiments on several benchmark datasets prove that our model can achieve higher performance without sacrificing efficiency

Transition-based Knowledge Graph Embedding with Relational Mapping Properties

PACLIC, (2014)

Cited by: 79|Views15
EI
Full Text
Bibtex
Weibo

Abstract

Many knowledge repositories nowadays contain billions of triplets, i.e. (head-entity, relationship, tail-entity), as relation instances. These triplets form a directed graph with entities as nodes and relationships as edges. However, this kind of symbolic and discrete storage structure makes it difficult for us to exploit the knowledge to...More

Code:

Data:

0
Introduction
  • Many knowledge repositories have been constructed either by experts with long-term funding (e.g. WordNet1 and OpenCyc2) or by crowds with collaborative contribution (e.g. Freebase3 and DBpedia4).
  • Each triplet, abbreviated as (h, r, t), is composed by two entities (i.e the head entity h and the tail entity t), and the relationship r between them.
  • These triplets can form a huge directed graph for each knowledge repository with millions of entities as nodes and thousands of relationships as edges.
Highlights
  • Many knowledge repositories have been constructed either by experts with long-term funding (e.g. WordNet1 and OpenCyc2) or by crowds with collaborative contribution (e.g. Freebase3 and DBpedia4)
  • We propose a superior model named TransM which aims at leveraging the structure information of the knowledge graph
  • Table 5 demonstrates that our model TransM outperforms the all the prior arts (i.e. the baseline model Unstructured (Bordes et al, 2014a), RESCAL (Nickel et al, 2011), SE (Bordes et al, 2011), SME (LINEAR) (Bordes et al, 2014a), SME (BILINEAR) (Bordes et al, 2014a), LFM (Jenatton et al, 2012) and the stateof-the-art TransE (Bordes et al, 2013a; Bordes et al, 2013b)) by evaluating them on the two benchmark datasets (i.e. WN18 and FB15K)
  • In order to conduct a fair comparison, the accuracy of Neural Tensor Network reported in Table 6 is same with the EV results in Figure 4 of (Socher et al, 2013)
  • The results of extensive experiments on several benchmark datasets prove that our model can achieve higher performance without sacrificing efficiency
Methods
  • Embedding the knowledge into low-dimensional space makes it much easier to conduct further AIrelated computing issues, such as link prediction and triplet classification (i.e. to discriminate whether a triplet (h, r, t) is correct or wrong).
  • Two latest related works (Bordes et al, 2013b; Socher et al, 2013) evaluate their model on the subsets of WordNet (WN) and Freebase (FB) data, respectively.
  • In order to conduct solid experiments, the authors compare the model with many related works including state-of-the-art and baseline.
  • 1: foreach r 2 R do 2: r := Uniform( p6 d ).
  • 3: r := Normalize(r) 4: end foreach
Results
  • The authors compare the model TransM with the stateof-the-art TransE and other models mentioned in (Bordes et al, 2013a) and (Bordes et al, 2014a) on the WN18 and FB15K.
  • Results show that TransM outperforms TransE when the authors choose L1 norm
  • These parameter combinations are adopted by the Triplet Classification task to search other parameters, which the authors will describe .
  • Table 8 shows the best performance of TransM and TransE when selecting L1 norm as the distance metric of the scoring functions.
Conclusion
  • TransM is a superior model that is expressive to represent the hierarchical and irreflexive characteristics and flexible to adapt various mapping properties of the knowledge triplets.
  • The authors provide an insight that the relational mapping properties of a knowledge graph can be exploited to enhance the model.
  • The authors look forward to applying Knowledge Graph Embedding to reinforce some other related fields, such as Relation Extraction from free texts (Weston et al, 2013) and Open Question Answering (Bordes et al, 2014b)
Tables
  • Table1: The scoring function and parameter complexity analysis for each related work. For all the models, we assume that there are a total of ne entities, nr relations (In most cases, ne nr.), and each entity is embedded into a ddimensional vector
  • Table2: Statistics of the datasets used for link prediction task
  • Table3: The detail results of link prediction between TransM and TransE on WN18 dataset when adopting L1 and L2 norm for the scoring function
  • Table4: The detail results of link prediction between TransM and TransE on FB15K dataset when adopting L1 and L2 norm for the scoring function
  • Table5: Link prediction results. We compared our proposed TransM with the state-of-the-art method (TransE) and other prior arts
  • Table6: The detail results of Filter Hit@10 (in %) on FB15K categorized by different mapping properties of relationship (M. stands for MANY)
  • Table7: Statistics of the datasets used for triplet classification task
  • Table8: The accuracy of triplet classification compared with the state-of-the-art method (TransE) and other prior arts
Download tables as Excel
Related work
  • Almost all the related works take efforts on embedding each entity or relationship into a lowdimensional continuous space. To achieve this goal, each of them defines a distinct scoring function fr(h, t) to measure the compatibility of a given triplet (h, r, t).

    Unstructured (Bordes et al, 2013b) is a naive model which just exploits the occurrence information of the head and the tail entities without considering the relationship between them. It defines a scoring function ||h t||, and obversely this model can not discriminate entity-pairs with different relationships. Therefore, Unstructured is commonly regarded as the baseline approach.

    Distance Model (SE) (Bordes et al, 2011) uses a pair of matrix, i.e (Wrh, Wrt), to represent the relationship r. The dissimilarity7 of a triplet (h, r, t) is calculate by the L1 distance of ||Wrhh Wrtt||. Even though the model takes the relationships into
Funding
  • This work is supported by National Program on Key Basic Research Project (973 Program) under Grant 2013CB329304, National Science Foundation of China (NSFC) under Grant No.61373075
Reference
  • Yoshua Bengio, Rjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155.
    Google ScholarLocate open access versionFindings
  • Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio, et al. 2011. Learning structured embeddings of knowledge bases. In AAAI.
    Google ScholarFindings
  • Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In International Conference on Artificial Intelligence and Statistics, pages 127–135.
    Google ScholarLocate open access versionFindings
  • Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013a. Irreflexive and hierarchical relations as translations. arXiv preprint arXiv:1304.7158.
    Findings
  • Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013b. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, pages 2787–2795.
    Google ScholarLocate open access versionFindings
  • Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2014a. A semantic matching energy function for learning with multi-relational data. Machine Learning, 94(2):233–259.
    Google ScholarLocate open access versionFindings
  • Antoine Bordes, Jason Weston, and Nicolas Usunier. 2014b. Open question answering with weakly supervised embedding models. CoRR, abs/1404.4326.
    Findings
  • Ulf Hermjakob, Eduard H Hovy, and Chin-Yew Lin. 2000. Knowledge-based question answering. In Proceedings of the Sixth World Multiconference on Systems, Cybernetics, and Informatics (SCI-2002).
    Google ScholarLocate open access versionFindings
  • Eric Huang, Richard Socher, Christopher Manning, and Andrew Ng. 2012. Improving word representations
    Google ScholarFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科