Concept Taxonomy

Conceptual hypernym prediction. Given an existing concept hierarchy tree (Taxonomy) 𝑇0 and a set of new concepts 𝐶, the goal is to predict its hypernym pa(𝑐) ∈ 𝑇0 for each new concept 𝑐 ∈ 𝐶 to expand the existing concept hierarchy tree.

MAG-full: This dataset is a concept hierarchy tree constructed by Microsoft Academic. It includes 355 808 concepts and 638 674 edges. Randomly sample 10% of the leaf nodes of the concept hierarchy tree as the validation set, randomly sample 10% as the test set, and the remaining 80% of the nodes and edges as the training set. Nodes in the validation and test sets are removed from the tree.

MAG-CS: This dataset is a concept hierarchy tree from the computer science domain of Microsoft Academic. It includes 24754 concepts and 42329 edges. The data set split method is similar to MAG-full, and the ratio of the number of concept nodes in the training set/validation set/test set is 8:1:1.

OAG-AI: This dataset is manually constructed by researchers in the field of artificial intelligence, covering five sub-fields of computer vision, data mining, information retrieval, machine learning, and natural language processing, including a total of 1335 concepts and 1283 edges. The data set split method is similar to MAG-full, and the ratio of the number of concept nodes in the training set/validation set/test set is 8:1:1. To evaluate the generalization ability of the models with respect to conceptual hyponym predictions, after training various models on MAG-CS, predictions and evaluations are performed on the OAG-AI dataset.

We match each new concept with the nodes in the existing concept hierarchy tree, and sort them according to their similarity. The evaluation indicators include: Hit@10, Mean Reciprocal Rank (MRR: the average value of the reciprocal of the ranking of real hypernyms).

Here, some of the latest conceptual hyponymy relationship discovery algorithms are selected for comparison.

• BiLinear: This method encodes new concept representations and candidate concept representations using a bilinear model for binary classification to determine whether the candidate concept is a hypernym of a new concept.

• TaxoExpan: It is one of the most advanced hyponymy relation discovery algorithms. It utilizes position-enhanced

Graph neural network measures the relationship between new concepts and candidate concept subgraphs, and utilizes InfoNCE loss function to enhance the robustness of the model.

• TaxoEnrich: This method first converts the existing hyponymy relations into natural language, and then the training model generates the initial representation of concepts; after that, the vertical relationship between concepts is encoded by LSTM, and the sibling relationship between concepts is encoded by attention mechanism; finally, a matching model between a new concept and a candidate position is used to calculate the matching score.

Code and Data: https://github.com/THUDM/OAG-taxo

We kindly request that any published research that makes use of this data cites the following papers.  

@inproceedings{zhang2024oag,
  title={OAG-bench: a human-curated benchmark for academic graph mining},
  author={Fanjin Zhang and Shijie Shi and Yifan Zhu and Bo Chen and Yukuo Cen and Jifan Yu and Yelin Chen and Lulu Wang and Qingfei Zhao and Yuqing Cheng and Tianyi Han and Yuwei An and Dan Zhang and Weng Lam Tam and Kun Cao and Yunhe Pang and Xinyu Guan and Huihui Yuan and Jian Song and Xiaoyan Li and Yuxiao Dong and Jie Tang},
  booktitle={Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  pages={6214--6225},
  year={2024}
}