FLEX-MPI: an MPI extension for supporting dynamic load balancing on heterogeneous non-dedicated systems

This paper introduces FLEX-MPI, a novel runtime approach for the dynamic load balancing of MPI-based SPMD applications running on heterogeneous platforms in the presence of dynamic external loads. To effectively balance the workload, FLEX-MPI monitors the actual performance of applications via hardware counters and the MPI profiling interface--with a negligible overhead and minimal code modifications. Our results show that by using this approach the execution time of an application may be significantly reduced.