# Arrow Matrix Decomposition: A Novel Approach for Communication-Efficient Sparse Matrix Multiplication

ACM SIGPLAN Symposium on Principles & Practice of Parallel Programming（2024）

Abstract

We propose a novel approach to iterated sparse matrix dense matrix
multiplication, a fundamental computational kernel in scientific computing and
graph neural network training. In cases where matrix sizes exceed the memory of
a single compute node, data transfer becomes a bottleneck. An approach based on
dense matrix multiplication algorithms leads to suboptimal scalability and
fails to exploit the sparsity in the problem. To address these challenges, we
propose decomposing the sparse matrix into a small number of highly structured
matrices called arrow matrices, which are connected by permutations. Our
approach enables communication-avoiding multiplications, achieving a polynomial
reduction in communication volume per iteration for matrices corresponding to
planar graphs and other minor-excluded families of graphs. Our evaluation
demonstrates that our approach outperforms a state-of-the-art method for sparse
matrix multiplication on matrices with hundreds of millions of rows, offering
near-linear strong and weak scaling.

MoreTranslated text

AI Read Science

Must-Reading Tree

Example

Generate MRT to find the research sequence of this paper

Chat Paper

Summary is being generated by the instructions you defined