AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
The results of extensive experiments on several benchmark datasets prove that our model can achieve higher performance without sacrificing efficiency
Transition-based Knowledge Graph Embedding with Relational Mapping Properties
Many knowledge repositories nowadays contain billions of triplets, i.e. (head-entity, relationship, tail-entity), as relation instances. These triplets form a directed graph with entities as nodes and relationships as edges. However, this kind of symbolic and discrete storage structure makes it difficult for us to exploit the knowledge to...More
PPT (Upload PPT)
- Many knowledge repositories have been constructed either by experts with long-term funding (e.g. WordNet1 and OpenCyc2) or by crowds with collaborative contribution (e.g. Freebase3 and DBpedia4).
- Each triplet, abbreviated as (h, r, t), is composed by two entities (i.e the head entity h and the tail entity t), and the relationship r between them.
- These triplets can form a huge directed graph for each knowledge repository with millions of entities as nodes and thousands of relationships as edges.
- Many knowledge repositories have been constructed either by experts with long-term funding (e.g. WordNet1 and OpenCyc2) or by crowds with collaborative contribution (e.g. Freebase3 and DBpedia4)
- We propose a superior model named TransM which aims at leveraging the structure information of the knowledge graph
- Table 5 demonstrates that our model TransM outperforms the all the prior arts (i.e. the baseline model Unstructured (Bordes et al, 2014a), RESCAL (Nickel et al, 2011), SE (Bordes et al, 2011), SME (LINEAR) (Bordes et al, 2014a), SME (BILINEAR) (Bordes et al, 2014a), LFM (Jenatton et al, 2012) and the stateof-the-art TransE (Bordes et al, 2013a; Bordes et al, 2013b)) by evaluating them on the two benchmark datasets (i.e. WN18 and FB15K)
- In order to conduct a fair comparison, the accuracy of Neural Tensor Network reported in Table 6 is same with the EV results in Figure 4 of (Socher et al, 2013)
- The results of extensive experiments on several benchmark datasets prove that our model can achieve higher performance without sacrificing efficiency
- Embedding the knowledge into low-dimensional space makes it much easier to conduct further AIrelated computing issues, such as link prediction and triplet classification (i.e. to discriminate whether a triplet (h, r, t) is correct or wrong).
- Two latest related works (Bordes et al, 2013b; Socher et al, 2013) evaluate their model on the subsets of WordNet (WN) and Freebase (FB) data, respectively.
- In order to conduct solid experiments, the authors compare the model with many related works including state-of-the-art and baseline.
- 1: foreach r 2 R do 2: r := Uniform( p6 d ).
- 3: r := Normalize(r) 4: end foreach
- The authors compare the model TransM with the stateof-the-art TransE and other models mentioned in (Bordes et al, 2013a) and (Bordes et al, 2014a) on the WN18 and FB15K.
- Results show that TransM outperforms TransE when the authors choose L1 norm
- These parameter combinations are adopted by the Triplet Classification task to search other parameters, which the authors will describe .
- Table 8 shows the best performance of TransM and TransE when selecting L1 norm as the distance metric of the scoring functions.
- TransM is a superior model that is expressive to represent the hierarchical and irreflexive characteristics and flexible to adapt various mapping properties of the knowledge triplets.
- The authors provide an insight that the relational mapping properties of a knowledge graph can be exploited to enhance the model.
- The authors look forward to applying Knowledge Graph Embedding to reinforce some other related fields, such as Relation Extraction from free texts (Weston et al, 2013) and Open Question Answering (Bordes et al, 2014b)
- Table1: The scoring function and parameter complexity analysis for each related work. For all the models, we assume that there are a total of ne entities, nr relations (In most cases, ne nr.), and each entity is embedded into a ddimensional vector
- Table2: Statistics of the datasets used for link prediction task
- Table3: The detail results of link prediction between TransM and TransE on WN18 dataset when adopting L1 and L2 norm for the scoring function
- Table4: The detail results of link prediction between TransM and TransE on FB15K dataset when adopting L1 and L2 norm for the scoring function
- Table5: Link prediction results. We compared our proposed TransM with the state-of-the-art method (TransE) and other prior arts
- Table6: The detail results of Filter Hit@10 (in %) on FB15K categorized by different mapping properties of relationship (M. stands for MANY)
- Table7: Statistics of the datasets used for triplet classification task
- Table8: The accuracy of triplet classification compared with the state-of-the-art method (TransE) and other prior arts
- Almost all the related works take efforts on embedding each entity or relationship into a lowdimensional continuous space. To achieve this goal, each of them defines a distinct scoring function fr(h, t) to measure the compatibility of a given triplet (h, r, t).
Unstructured (Bordes et al, 2013b) is a naive model which just exploits the occurrence information of the head and the tail entities without considering the relationship between them. It defines a scoring function ||h t||, and obversely this model can not discriminate entity-pairs with different relationships. Therefore, Unstructured is commonly regarded as the baseline approach.
Distance Model (SE) (Bordes et al, 2011) uses a pair of matrix, i.e (Wrh, Wrt), to represent the relationship r. The dissimilarity7 of a triplet (h, r, t) is calculate by the L1 distance of ||Wrhh Wrtt||. Even though the model takes the relationships into
- This work is supported by National Program on Key Basic Research Project (973 Program) under Grant 2013CB329304, National Science Foundation of China (NSFC) under Grant No.61373075
- Yoshua Bengio, Rjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155.
- Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio, et al. 2011. Learning structured embeddings of knowledge bases. In AAAI.
- Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In International Conference on Artificial Intelligence and Statistics, pages 127–135.
- Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013a. Irreflexive and hierarchical relations as translations. arXiv preprint arXiv:1304.7158.
- Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013b. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, pages 2787–2795.
- Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2014a. A semantic matching energy function for learning with multi-relational data. Machine Learning, 94(2):233–259.
- Antoine Bordes, Jason Weston, and Nicolas Usunier. 2014b. Open question answering with weakly supervised embedding models. CoRR, abs/1404.4326.
- Ulf Hermjakob, Eduard H Hovy, and Chin-Yew Lin. 2000. Knowledge-based question answering. In Proceedings of the Sixth World Multiconference on Systems, Cybernetics, and Informatics (SCI-2002).
- Eric Huang, Richard Socher, Christopher Manning, and Andrew Ng. 2012. Improving word representations