Bi-objective cloud resource management for dependent tasks using Q-learning and NSGA-3

Efficient management of cloud resources for more resource utilization on the one hand, and reducing the makespan on the other hand, has always been an important research issue in cloud environment. Since the proper allocation of cloud resources is an optimization problem, the learning-based and population-based meta-heuristic methods are used for this purpose. Most learning methods have scalability problem and may not converge to the optimal solution as the problem space becomes larger, and population-based optimization methods usually need a lot of iterations. The proposed method of this paper aims to overcome the limitations of these two techniques and uses their advantages to increase cloud resource utilization and improve the execution time. A Pareto-based algorithm is also exploited to solve this bi-objective problems. Since increasing the size of population leads to increase the convergence time, a proper learning-based selecting method of population is also utilized to reduce the number of iterations. The proposed method of this paper, called multi-agent bi-objective cloud resource management for dependent tasks using Q-learning and NSGA-3 (BCRN), employs an improved version of Q-learning to reduce the makespan and enhance resource utilization. To overcome the scalability problem of Q-learning, the number of states and actions are reduced in the BCRN, which reduces the complexity of the learning process and leads to better convergence time of the learning process. Two learning agents are also utilized in the BCRN, each of which seeks to improve the objectives of the problem. The NSGA-3 (non-deterministic sorting genetic algorithm-3) algorithm is used as to address the bi-objective problem. In the NSGA-3 algorithm, the initial population is often generated randomly, which leads to more convergence time of the learning process. Using a modified bi-objective Q-learning model, the initial population of the BCRN is generated considering the both objectives of the method to reduce the convergence time of the NSGA-3 algorithm. The long-term learning process attempts to find the best mapping between resources and tasks, and avoid local optima. The results of empirical experiments indicate that the proposed method reduces the makespan, and causes more efficient use of cloud resources.