Scheduling Tightly Coupled Parallel Jobs With Runtime-Extended Tasks on Distributed Resources

Gang scheduling is an effective and efficient approach to schedule jobs with tightly coupled parallel tasks. However, it becomes particularly challenging when the gang jobs feature a varying mean arrival rate, as well as a varying degree of parallelism. An additional challenge is imposed in case the result of a gang job is below a defined Quality of Service (QoS) threshold and thus it is not acceptable. In this case, the execution time of all of the parallel component tasks of the gang job should be dynamically extended at runtime, in order to perform more calculations and produce an acceptable result. Consequently, it is important to examine the performance of gang scheduling techniques under these workload characteristics. To this end, in this paper we investigate in a distributed environment the performance of two well-known gang scheduling policies for gang jobs with a varying mean arrival rate, a varying degree of parallelism, and runtime-extended execution times. Our goal is to evaluate the relative performance of the employed scheduling strategies in different cases of the probability for a gang job to have its tasks dynamically extended, under different workload conditions. The results of the simulation experiments reveal that the performance of the scheduling approaches depends on the probability for a gang job to have its execution time extended.