A Reinforcement Learning-Based Population Hyper-Heuristic for Energy-Efficient Cloud Workflow Scheduling Problem

As cloud computing undergoes rapid development, cloud workflow scheduling has gained significant attention as a critical area of research. However, the substantial energy consumption of cloud data centers in handling diverse workflow applications highlights the need for effective energy optimization strategies to promote green computing. This research investigates an energy-efficient cloud workflow scheduling problem that explores the inherent relationship between scheduling schemes, host load, and energy consumption. To tackle this problem, a reinforcement learning-based population hyper-heuristic algorithm is proposed to provide an effective solution. Specifically, six low-level heuristics (LLHs) tailored to the characteristics of the problem are developed and refined. Meanwhile, Q-learning is employed as a high-level strategy for the intelligent selection of LLHs. Moreover, an adaptive local search strategy is introduced to improve inferior individuals. Additionally, an experience-driven task-resource mapping approach is employed to realize efficient resource allocation. To mitigate the challenge of infeasible solutions, a region-constrained insertion mechanism is devised to effectively minimize their occurrence. Subsequently, a solution feasibility reconstruction strategy is introduced to adjust and optimize any remaining infeasible solutions. Extensive experiments on various workflow applications of different scales and types demonstrate the superiority of the proposed algorithm over existing approaches in terms of energy optimization and exhibit significant practical application potential.