Entity Tagging

Subtask 1: Scholars' research interest prediction.

Given a collection of papers, a co-author network, research interest labels, and the research interests of partial scholars are known, the goal is to predict the top-K research interests of scholars in the test set.

Subtask 2: Paper topic classification

Given a collection of papers, a paper-citation network, k topics, and the topics of partial papers are known, the goal is to assign the most suitable topic for papers in the test set.

OPEDAC-2017: This dataset includes manually-labeled research interest labels of 11357 scholars and the papers of these scholars. Data from 6000 scholars are used for training and the remaining data are used for a test set.

DBLP-Paper-Topic: This dataset is based on DBLP paper citation network. The topic of each paper is determined by its venue. There are 9 topics in total. This dataset includes 992606 papers. 634273 papers published no later than 2012 are used for training. 250555 papers published in 2013 and 2014 are used for validation. 107778 papers published after 2015 are used for testing.

For scholars' research interest prediction, we evaluate the matching ratio of predicted labels and ground truths.

where N is the number of samples of the test dataset in task 2, Ti is a set of the ith scholar's research interests calculated by each team and Ti* is a set of the ith given scholar's research interest. Here, |Ti*|=3.

For paper topic classification, we evaluation the accuracy of multi-class classification.

LSI: This method calculates the similarity of the titles of the papers published by the author of the test set and the author of the training set, and uses the interest label of the author of the training set and the weighted average similarity scores as the interest label of the author of the test set. When calculating the text similarity, the paper text is transformed into a bag of words model and TF-IDF representation, and the LSI model is used for dimensionality reduction, and then the text similarity is calculated.

Sentence-BERT: This method is similar to the LSI method, the difference lies in the calculation of text similarity. This method uses Sentence-BERT to encode the paper text representation and uses the encoded vector to calculate the text similarity.

ACA1: This method uses the title of the paper published by the scholar, the citation information of the paper, and the venue published by the scholar to predict the interest label of the scholar. The author of the test set and the author of the training set are calculated according to different attributes, and then the author’s interest label of the test set author is calculated according to the author’s similarity and the weight of the interest label of the author of the training set.

SGC: SGC removes the nonlinear transformation of the graph convolution layer, and decouples the node feature propagation and feature nonlinear transformation on the graph. SGC can first calculate the result of multi-stage feature propagation and then train the node classifier.

SIGN: SIGN uses multiple graph convolution filters of different sizes to generate node representations, which avoids the complexity of node sampling and subgraph sampling.

GraphSAGE: Sample the surrounding neighbor nodes for each node, and design multiple aggregation methods to aggregate the node representations of multi-hop neighbors. The objective function is the cross-entropy loss of node representations and node labels.

Code and Data:

https://github.com/THUDM/oag-author-tagging  

https://github.com/QingFei1/dpv12_nodeclassification

We kindly request that any published research that makes use of this data cites the following papers.  

@inproceedings{zhang2024oag,
  title={OAG-bench: a human-curated benchmark for academic graph mining},
  author={Fanjin Zhang and Shijie Shi and Yifan Zhu and Bo Chen and Yukuo Cen and Jifan Yu and Yelin Chen and Lulu Wang and Qingfei Zhao and Yuqing Cheng and Tianyi Han and Yuwei An and Dan Zhang and Weng Lam Tam and Kun Cao and Yunhe Pang and Xinyu Guan and Huihui Yuan and Jian Song and Xiaoyan Li and Yuxiao Dong and Jie Tang},
  booktitle={Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  pages={6214--6225},
  year={2024}
}