WidePipe: High-Throughput Deep Learning Inference System on a Cluster of Neural Processing Units
2021 IEEE 39th International Conference on Computer Design (ICCD)(2021)
摘要
The wide application of machine learning technology promotes the generation of ML-as-a-Service(MLaaS), which is a serverless computing paradigm for rapidly deploying a trained model as a serving. However, it is a challenge to design an inference system that is capable of coping with large traffic for low latency and heterogeneous neural networks. It is difficult to adaptively configure multilevel ...
更多查看译文
关键词
Performance evaluation,Neural networks,Serverless computing,Reinforcement learning,Parallel processing,Throughput,Time factors
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要