Masked Graph Autoencoder with Non-discrete Bandwidths
CoRR(2024)
摘要
Masked graph autoencoders have emerged as a powerful graph self-supervised
learning method that has yet to be fully explored. In this paper, we unveil
that the existing discrete edge masking and binary link reconstruction
strategies are insufficient to learn topologically informative representations,
from the perspective of message propagation on graph neural networks. These
limitations include blocking message flows, vulnerability to over-smoothness,
and suboptimal neighborhood discriminability. Inspired by these understandings,
we explore non-discrete edge masks, which are sampled from a continuous and
dispersive probability distribution instead of the discrete Bernoulli
distribution. These masks restrict the amount of output messages for each edge,
referred to as "bandwidths". We propose a novel, informative, and effective
topological masked graph autoencoder using bandwidth masking and a layer-wise
bandwidth prediction objective. We demonstrate its powerful graph topological
learning ability both theoretically and empirically. Our proposed framework
outperforms representative baselines in both self-supervised link prediction
(improving the discrete edge reconstructors by at most 20
classification on numerous datasets, solely with a structure-learning pretext.
Our implementation is available at https://github.com/Newiz430/Bandana.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要