Moirai: Towards Optimal Placement for Distributed Inference on Heterogeneous Devices
CoRR(2023)
摘要
The escalating size of Deep Neural Networks (DNNs) has spurred a growing
research interest in hosting and serving DNN models across multiple devices. A
number of studies have been reported to partition a DNN model across devices,
providing device placement solutions. The methods appeared in the literature,
however, either suffer from poor placement performance due to the exponential
search space or miss an optimal placement as a consequence of the reduced
search space with limited heuristics. Moreover, these methods have ignored the
runtime inter-operator optimization of a computation graph when coarsening the
graph, which degrades the end-to-end inference performance. This paper presents
Moirai that better exploits runtime inter-operator fusion in a model to render
a coarsened computation graph, reducing the search space while maintaining the
inter-operator optimization provided by inference backends. Moirai also
generalizes the device placement algorithm from multiple perspectives by
considering inference constraints and device heterogeneity.Extensive
experimental evaluation with 11 large DNNs demonstrates that Moirai outperforms
the state-of-the-art counterparts, i.e., Placeto, m-SCT, and GETF, up to
4.28$\times$ in reduction of the end-to-end inference latency. Moirai code is
anonymously released at \url{https://github.com/moirai-placement/moirai}.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要