Deep Reinforcement Learning-Based Algorithms Selectors for the Resource Scheduling in Hierarchical Cloud Computing

Cloud computing environment is becoming increasingly complex due to its large-scale information growth and increasing heterogeneity of computing resources. Hierarchical Cloud computing dividing the system into multi-levels with multiple subsystems to support the adaptability to abundant requests from users has been widely applied and brings great challenges to resource scheduling. It is critical to find an effective way to address the complex scheduling problems in hierarchical Cloud computing, whose scenarios and optimization objectives often change with the types of subsystems. In this paper, we propose a scheduling framework to select the scheduling algorithms (SFSSA) for different scheduling scenarios considering no algorithm well suitable to all scenarios. To concretize SFSSA, we propose deep learning-based algorithms selectors (DLS) trained by labeled data and deep reinforcement learning-based algorithms selectors (DRLS) trained by feedback from dynamic scenarios to complete the algorithms selection regarding the scheduling algorithms as selectable tools. Then, we apply strategies including pre-trained model, long experience reply and joint training to improve the performance of DRLS. To enable the quantitative comparison of selectors, we introduce a weighted cost model for the trade-off between solution and complexity. Through multiple sets of experiments in hierarchical Cloud computing with multi subsystems for five types of scheduling problems and varying weights of cost, we demonstrate DLS and DRLS outperform baseline strategies. Compared with random selector, greedy selector, round-robin selector, single best selector, virtual best selector and single fast selector, DLS reduces the cost by 47.4%, 46.1%, 33.9%, 47.9%, 19.3%, 18.8% under stable parameter ranges, and DRLS reduces the cost by 41.1%, 40.6%, 11.7%, 42.3%, 11.5%, 12.5% in dynamic scenarios respectively. In experiments, we also validate DRLS has stronger adaptability than DLS in dynamic scheduling scenarios and DRLS using all of strategies achieves the best performance.