Simulating Burst Buffer for an Efficient Data Placement Policies Selection

Data access has become the central issue in the era of Big Data and IO-intensive applications. This pressure on storage systems is greater in HPC, as supercomputers and their high computing capacity to handle compute-intensive applications should not be delayed by the access to data. Burst buffers play a major role in the acceleration of data access, with their multi-tiered storage architectures featuring different technologies and performance characteristics. However, data placement and management across the tiers impacts the delivered performance. In this work, we develop a simulator that is cost-effective in computation and memory footprint, capable of being integrated between a file profiler (FiLiP) and a burst buffer (SBB) on a datanode in an industrial environment. It uses the predicted files access schemes from the history extracted by FiLiP, and simulates several data placement policies for different hardware configurations executing the predicted workload. The simulation generates hints for the burst buffer to anticipate the selection of the most appropriate data management policy. The simulator is well calibrated based on real measures and its use for three scientific reference applications allowed an increase of the hit rate by a factor of 3.31 compared to the initial placement policy for the LQCD application.