GTC: GNN-Transformer Co-contrastive Learning for Self-supervised Heterogeneous Graph Representation
CoRR(2024)
摘要
Graph Neural Networks (GNNs) have emerged as the most powerful weapon for
various graph tasks due to the message-passing mechanism's great local
information aggregation ability. However, over-smoothing has always hindered
GNNs from going deeper and capturing multi-hop neighbors. Unlike GNNs,
Transformers can model global information and multi-hop interactions via
multi-head self-attention and a proper Transformer structure can show more
immunity to the over-smoothing problem. So, can we propose a novel framework to
combine GNN and Transformer, integrating both GNN's local information
aggregation and Transformer's global information modeling ability to eliminate
the over-smoothing problem? To realize this, this paper proposes a
collaborative learning scheme for GNN-Transformer and constructs GTC
architecture. GTC leverages the GNN and Transformer branch to encode node
information from different views respectively, and establishes contrastive
learning tasks based on the encoded cross-view information to realize
self-supervised heterogeneous graph representation. For the Transformer branch,
we propose Metapath-aware Hop2Token and CG-Hetphormer, which can cooperate with
GNN to attentively encode neighborhood information from different levels. As
far as we know, this is the first attempt in the field of graph representation
learning to utilize both GNN and Transformer to collaboratively capture
different view information and conduct cross-view contrastive learning. The
experiments on real datasets show that GTC exhibits superior performance
compared with state-of-the-art methods. Codes can be available at
https://github.com/PHD-lanyu/GTC.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要