CUTTANA: Scalable Graph Partitioning for Faster Distributed Graph Databases and Analytics
arxiv(2023)
摘要
Graph partitioning plays a pivotal role in various distributed graph
processing applications, including graph analytics, graph neural network
training, and distributed graph databases. Graphs that require distributed
settings are often too large to fit in the main memory of a single machine.
This challenge renders traditional in-memory graph partitioners infeasible,
leading to the emergence of streaming solutions. Streaming partitioners produce
lower-quality partitions because they work from partial information and must
make premature decisions before they have a complete view of a vertex's
neighborhood. We introduce CUTTANA, a streaming graph partitioner that
partitions massive graphs (Web/Twitter scale) with superior quality compared to
existing streaming solutions. CUTTANA uses a novel buffering technique that
prevents the premature assignment of vertices to partitions and a scalable
coarsening and refinement technique that enables a complete graph view,
improving the intermediate assignment made by a streaming partitioner. We
implemented a parallel version for CUTTANA that offers nearly the same
partitioning latency as existing streaming partitioners.
Our experimental analysis shows that CUTTANA consistently yields better
partitioning quality than existing state-of-the-art streaming vertex
partitioners in terms of both edge-cut and communication volume metrics. We
also evaluate the workload latencies that result from using CUTTANA and other
partitioners in distributed graph analytics and databases. CUTTANA outperforms
the other methods in most scenarios (algorithms, datasets). In analytics
applications, CUTTANA improves runtime performance by up to 59
various streaming partitioners (HDRF, Fennel, Ginger, HeiStream). In graph
database tasks, CUTTANA results in higher query throughput by up to 23
without hurting tail latency.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要