DistFNE: A Distributed-Memory Algorithm for Force-Directed Node Embedding

We develop a distributed-memory algorithm to embed nodes of a graph into a low-dimensional vector space. Our distributed algorithm, called DistFNE, is based on a force-directed layout that maximizes attraction among neighboring vertices and repulsion among distant ones. DistFNE utilizes large minibatches in stochastic gradient descent (SGD) to scale node embeddings to thousands of processors without compromising the quality of the embeddings. DistFNE optimizes memory usage and inter-process communication through a configurable push-pull strategy. We employ asynchronous MPI communication to overlap communication and computation. These combined techniques made DistFNE scalable to 32,768 cores of supercomputers to generate embeddings of graphs with billions of edges.