DeepScale: Forecast-Based Vertical Scaling in Cloud-Native MLOps

Deep learning training workloads are inherently non-stationary and exhibit rapid variability, making static compute reservations in Cloud-native Machine Learning Operations (MLOps) pipelines either wasteful (due to over-provisioning) or risky (due to under-provisioning). We present DeepScale, a forecast-driven, high-frequency, in-place vertical pod autoscaling (VPA) framework that proactively right-sizes training pods without evictions or downtime. DeepScale continuously ingests per-pod CPU telemetry, forecasts near-term demand using time-series models (XGBoost, LSTM, N-BEATS), and applies policy-aware safety margins and buffers to produce robust resource recommendations. By leveraging in-place VPA in recent Kubernetes releases, resources are updated live with sub-minute convergence. Evaluated on three representative training pipelines, i.e., DNN (MNIST), CNN (CIFAR-10), and LSTM (IMDB), DeepScale reduces requested vCPU-hours by 38–91% over static and current in-place VPA baselines, hence lowering the cost proportionally. These results indicate that coupling predictive control with in-place vertical scaling yields a practical, self-optimizing, and cost-aware autoscaling solution for deep-learning training workloads.