Knowledge Graph Completion with Information Adaptation and Refinement

At present, knowledge graph completion (KGC) is mainly divided into structure-based methods and language-based methods, which characterize the structural information and semantic information of knowledge graphs, respectively. Though existing works have developed methods to integrate both information, we argue their end-to-end training manner suffers discrepancy, compatibility, resources redundancy issues. Therefore, we propose a novel two-stage training paradigm for tackling KGC task, i.e. information adaptation and refinement (KGCIAR). Specifically, KGCIAR has two stages, 1) adaptation and 2) refinement. In the adaptation stage, we fine-tune the PLM with the input of descriptive information and supervised by the KG structural information. In the second refinement stage, we freeze the adapted PLM model and infer the description embeddings of entities and relations. Then, those embeddings are leveraged as the entity/relation initial embeddings. Finally, we train a lightweight KGC model. Moreover, we devise two novel objectives for knowledge adaptation, which are self-supervised adaptation and structure-aware contrastive adaptation. Furthermore, we systematically compare the performance of different lightweight KGC models for information refinement. The experiments on KGC task and various variants analyses demonstrate that KGCIAR is effective in harnessing both structure and language information in KG.