Identifying architectural design decisions for achieving green ML serving
CoRR(2024)
摘要
The growing use of large machine learning models highlights concerns about
their increasing computational demands. While the energy consumption of their
training phase has received attention, fewer works have considered the
inference phase. For ML inference, the binding of ML models to the ML system
for user access, known as ML serving, is a critical yet understudied step for
achieving efficiency in ML applications.
We examine the literature in ML architectural design decisions and Green AI,
with a special focus on ML serving. The aim is to analyze ML serving
architectural design decisions for the purpose of understanding and identifying
them with respect to quality characteristics from the point of view of
researchers and practitioners in the context of ML serving literature.
Our results (i) identify ML serving architectural design decisions along with
their corresponding components and associated technological stack, and (ii)
provide an overview of the quality characteristics studied in the literature,
including energy efficiency.
This preliminary study is the first step in our goal to achieve green ML
serving. Our analysis may aid ML researchers and practitioners in making
green-aware architecture design decisions when serving their models.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要