Spatio-Temporal Resource Control for Cloud-Native GPU Provisioning.

Modern cloud platforms, such as Kubernetes, provide a service-oriented resource abstraction for explicit Quality of Service (QoS) provisioning, which guarantees resource reservations and supports resource elasticity. In regards to CPU resources, tenants can explicitly specify a guaranteed base demand and an upper bound for resources. Thus, it is desirable that GPU resource provisioning should be analogous to mature cloud-native CPU resource abstraction and can support familiar QoS classes, such as Guaranteed, Burstable, and BestEffort. Although existing research efforts have primarily focused on maximizing GPU utilization by exploiting profiling, capabilities for guaranteed reservation, precise throttling, and elastic bursting are essential to support cloud-native GPU provisioning. It is challenging to provide such features due to the GPU's asynchronous and non-preemptive characteristics. To address this issue, we introduce a spatio-temporal provisioning framework that ensures both resource guarantee and elasticity for GPUs. We define a resource model to control both spatial and temporal dimensions of GPU resources and support familiar QoS classes. Our framework features a per-container Agent for transparent resource accounting and local quota enforcement, and the central Multi-Tenant Arbitrator for global, partition-aware fair scheduling. Performance evaluation demonstrates that our framework can provide accurate GPU resource guarantee and manages dynamic mixed-QoS workloads by honoring reserved GPU resources during contention while allowing tenants to elastically burst into idle capacity up to their limit.