Category-Centric Initialization for Knowledge Graph Embedding

This paper proposes a category-centric initialization that introduces prior knowledge for knowledge graph embedding (KGE) at a low cost. KGE is a technology that maps symbols to embeddings to utilize large-scale knowledge graphs, and it has been widely applied because of its simplicity and efficiency. However, the initialization challenge with this technology has long been overlooked. This critical issue has implications for the training cost, the stability of training, and even the final performance of the model. KGE predominantly utilizes random initialization, which overlooks the wealth of prior knowledge embedded within knowledge graphs. To counteract this, pre-training initialization has been introduced as a way to utilize the prior knowledge. While this strategy can lead to enhanced model performance and quicker convergence rates, it increases computational demands and restricts application breadth. To address these challenges, we propose a novel initialization called category-centric initialization (CCI). CCI is designed to be universally applicable across any scenario involving the training of KGE models from scratch. It utilizes the weighted sum of the category embedding and the random embedding as the initial embedding of entities. By integrating explicit category information into the random initialization, CCI effectively utilizes prior knowledge while avoiding excessive computational cost. The results of experiments demonstrate that the proposed method can effectively reduce the training cost of advanced KGE models without degrading the final performance. Additionally, the results of experiments without category information show that our method can be applied in scenarios where explicit categories are not given to entities.