Enabling Efficient Distributed Graph Neural Network Acceleration with Near Memory Processing

Distributed Graph Neural Networks (GNNs) require efficient handling of both fine-grained memory accesses and cross memory-device communication, particularly when scaling to large graphs in multimedia tasks. However, existing acceleration solutions fail to adequately address low bandwidth utilization and scalability across different models and graph sizes. In this paper, we present OptGNN, a scalable heterogeneous distributed architecture tailored for GNNs. OptGNN addresses the challenges of fine-grained memory access by incorporating a near-memory processing mechanism, which improves internal bandwidth utilization. To optimize external communication, we introduce data packing and scheduling strategies that enhance cross memory-device data transfer efficiency. OptGNN achieves 5.7x performance improvement over baseline distributed GNN acceleration methods and 1.29x performance improvement over SOTA distributed GNN acceleration architecture CLAY. Additionally, the system is designed to support various GNN models and large-scale graphs while ensuring load balancing and high hardware utilization.