EMC+: an Opportunistic Elasticity Method for Improving System Throughput and CPU Utilization in Cloud Data Centers

ABSTRACT Multicore architectures have become the prevailing processor design for general‐purpose computing systems, enabling high performance across a wide range of commercial platforms. Technological advances have made it possible to integrate hundreds of cores on a single package, providing unprecedented computational capacity. In cloud data centers, efficiently exploiting this growing core count becomes crucial for improving resource efficiency, lowering operational costs, and boosting revenue. To achieve this, servers usually run multiple colocated cloud services alongside diverse workloads. However, fully utilizing all available cores while enforcing quality‐of‐service (QoS) constraints for critical services remains a major challenge. This paper introduces a novel OS‐level approach designed to maximize CPU usage in multicore servers through the opportunistic acceleration of elastic HPC workloads. These workloads are capable of dynamically adjusting the number of active worker threads/processes at runtime. Our proposal, referred to as EMC+, is an OS‐driven elasticity manager for container‐based environments that continuously estimates idle core cycles left by regular (inelastic) applications, and reallocates idle cores to elastic ones, even during short time intervals. The new EMC+ proposal builds on our previous work and broadens the scope of OS‐driven elasticity through several key contributions: the design of a new kernel‐level container‐management framework, efficient operation on large‐scale multicore platforms, the integration of QoS concerns into elasticity exploitation, and a comprehensive experimental analysis using diverse workloads that combine cloud services and HPC applications based on different parallel programming models. Experimental results show that EMC+ speeds up elastic workloads by up to 2.3 (1.5 on average), while having minimal impact on the performance and QoS of colocated inelastic applications. Compared to the unmodified Linux kernel, which does not exploit opportunistic elasticity, EMC+ boosts average system throughput by 19%, while degrading inelastic application performance by only 2.2% on average, and increasing the occurrence of observation intervals with QoS violations by 3.1% on average.