Dynamic Memory Provisioning on Disaggregated HPC Systems.

Disaggregated memory is under investigation as a way to break the rigid boundaries between node memory hierarchies in order to provide memory as a system-wide pooled resource. The resource manager allocates the system’s disaggregated memory to jobs, based on the memory requirements defined by the user at job submission time. It is hard for the user to know the job’s precise peak memory footprint, and prior work has shown that users have an incentive to overestimate their needs. This overestimation leads to a significant overallocation of memory, and the majority of the physical memory in the system is wasted. This paper presents a way to reclaim much of this overallocated memory. We extend the Slurm job scheduler to dynamically reallocate memory, according to the job’s current memory footprint. We enhance an existing Slurm simulator to model this situation and combine publicly available traces to model an HPC system on up to 1490 nodes. Our results show that the dynamic memory provisioning approach increases the throughput per dollar by up to 38%, compared to a system with static allocation of disaggregated memory.