GPU Cluster Dynamics: Insights from Alibaba's 2023 Trace Release

Abstract In this paper, we present a comprehensive analysis of GPU cluster traces from Alibaba, released in 2023, with a focus on unraveling the intricacies of node and pod configurations and their associated metrics. By dissecting the configurations of 1,523 nodes, predominantly GPU-based, we identify a wide array of configurations and GPU models, highlighting a balanced interplay between CPU cores and RAM across all nodes while also noting a decoupling between CPU/RAM and GPU-specific metrics in GPU nodes, indicating a nuanced approach to workload placement. Our investigation extends to an analysis of 8,152 pods, revealing diverse configurations and a significant prevalence of latency-sensitive pods, indicative of a cluster geared towards timely resource provisioning. The presence of Failed and Pending pods, however, underscores the complexity and challenges inherent in scheduling efficiency and resource allocation. By delving into both node and pod configurations and metrics, this study sheds light on the operational characteristics and workload patterns within the cluster. The insights gleaned from this analysis are invaluable for researchers, system designers, and operators, providing a nuanced understanding of the cluster's workings. This, in turn, lays the groundwork for future research and optimization efforts, ultimately contributing to the advancement of GPU cluster management and operation.