Self-Supervised Learning with Atom Replacement for Catalyst Energy Prediction by Graph Neural Networks

Recently, material search utilizing the material informatics (MI) technique is considered to be an efficient approach to discovering new materials. Energy or environment-related materials are deeply related to chemical reactions, and the reactivity of materials is determined by the electronic energy (or internal energy) of materials and reactant systems. The electronic energy is often calculated by first-principles calculation which is based on the Schrödinger equation. Among them, a method called density functional theory (DFT) is widely used nowadays. Since the computational cost of the DFT calculation is relatively high, many graph neural networks (GNNs) for energy prediction have been proposed to obtain the energy values quickly. However, it is difficult to obtain a large amount of labeled data for GNNs training, because a large amount of the DFT data is necessary for training. Although self-supervised learning (SSL) methods using unlabeled data, i.e., no DFT energy information is needed, have been proposed to improve the GNNs’ energy prediction accuracy, existing SSL methods use a non-existent atom of “mask”; this may lead to the inefficient GNNs training. In this work, we propose a mask-less SSL method by replacing some atoms in the target material systems, aiming to improve the energy prediction accuracy of GNNs. Our target systems are catalysts, and here the atoms in unlabeled catalysts are replaced with other actual existing atoms. Then, GNNs predict whether the atoms have been replaced or not for all atoms composing the material. After that, GNNs pre-trained by the proposed SSL are fine-tuned by downstream tasks. We demonstrate the superiority of our proposed method on the Open Catalyst dataset and Poisoned Catalyst dataset using the energy prediction GNNs, CGCNN and PaiNN. The GNNs pre-trained by the proposed SSL achieve the best mean absolute error of predicted energy rather than that of the existing mask-based SSL in all experimental conditions.