Scheduling distributed applications modeled as directed, acyclic task graphs to run on heterogeneous compute networks is a fundamental (NP-Hard) problem in distributed computing for which many heuristic algorithms have been proposed over the past decades. Many of these algorithms fall under the list-scheduling paradigm, whereby the algorithm first computes priorities for the tasks and then schedules them greedily to the compute node that minimizes some cost function. Thus, many algorithms differ from each other only in a few key components (e.g., the way they prioritize tasks, their cost functions, where the algorithms consider inserting tasks into a partially complete schedule, etc.). We propose a generalized list-scheduling algorithm that allows mixing and matching different task prioritization and greedy node selection schemes to produce 72 unique algorithms. We benchmark these algorithms on 20 datasets to study the individual and combined effects of different algorithmic components on performance and runtime. We show that while every algorithmic component we consider is well-established in the literature, many of their combinations which have never before been considered are pareto-optimal with respect to average performance and runtime. We also report interesting interactions between algorithmic components and dataset features, suggesting that performance is highly application-dependent.
With current trends in Data Center power consumption, and the increase of AI oriented Data Centers with power hungry GPU nodes, the management and control of power consumption has become one of the key points in keeping resources in check. Power is a limited resource as is the number of available CPUs or GPUs. However, power is a dynamic resource and a computational node can consume more or less power depending on the workload and hardware settings (CPU and GPU frequency for example). There are multiple potential strategies to manage the total available power in a cluster but all of them have a certain impact on job performance and on workload performance. In this paper, we present the EAR hard-powercap strategy and the simulation framework we have implemented to evaluate, as accurately as possible, the performance impact of the EAR hard-powercap strategy when running large workloads in big Data Centers. The performance results will be compared against a static powercap strategy. Since power and performance relationship is not constant, we have introduced the concept of power profiles as part of the simulations and several use cases, for each workload, have been simulated. Both strategies have been evaluated using workload trace files + power usage profiles and simulated in an evaluation framework composed by a Job scheduler simulator (Batsim + Batsched) + ClusterSim (N Node manager simulated in a single process) + EAR System power manager. Results show a substantial reduction on both wait time and execution time, especially as the cluster’s power constraints increase when using EAR’s strategy.
Co-Scheduling jobs in High Performance Computing (HPC) systems offers significant potential to improve system throughput and energy efficiency. However, resource contention in shared node resources can introduce performance degradation, leading to job slowdowns and counteracting these benefits. To address this challenge, sophisticated co-scheduling algorithms must be developed, requiring a good understanding of the submitted applications to make informed scheduling decisions. In this work, we classify and present a number of performance models that can be leveraged to support advanced co-scheduling strategies. The methods focus on either assigning specific tags to applications or predicting their potential speedup or slowdown when co-executed with other workloads. To achieve this, we explore both empirical approaches and Machine Learning-based techniques, assessing their respective benefits and limitations. Furthermore, we discuss key trade-offs that arise when selecting and building the most suitable model for beneficial co-location prediction in HPC environments. Finally, we provide preliminary results demonstrating the effectiveness of each model through representative examples across multiple model categories.
Most current job schedulers determine the job schedule and resources based on first-come-first-served with backfilling policy. However, the majority of the current job schedulers do not consider the communication pattern of a job. This may result in performance slowdown due to job interference. We propose and implement Combal (Communication Balanced) algorithm in the SLURM job scheduler. Combal allocates resources based on the communication profile of a job and the current cluster state. Combal aims to reduce the communication cost by allocating frequently communicating processes on compute nodes of the same leaf switch of a fat-tree. We compare this with TopoMatch algorithm using real job logs from the Intrepid, Mira, and Theta supercomputers. We observed up to 20
Efficient resource management and scheduling have been persistent challenges since the early days of computing and remain crucial today. The widespread adoption of containers managed by orchestrators like Kubernetes has introduced new dimensions to these challenges. Despite their lightweight nature, containers still suffer from inefficiencies due to over-provisioning and long-running workloads allocating resources potentially forever. Existing scheduling techniques are not enough to meet these demands, and there is a growing need for orchestration and scheduling policies that support advanced preemption, migration, and fault tolerance. Well-established container Checkpoint/Restore (C/R) mechanisms, implemented through tools like CRIU, offer a promising solution for improving resource scheduling efficiency. However, these mechanisms remain only partially integrated with Kubernetes. In this paper, we propose a novel Interruption-Aware Scheduling Strategy that incorporates transparent C/R with the Kubernetes scheduler. We discuss the current C/R mechanisms, outline the key design choices involved in integrating with the Kubernetes scheduler, and explore associated challenges and open problems. We further discuss potential solutions to these challenges, offering a path toward more efficient resource management to better meet the needs of today’s computational landscape.
