A Two-stage Learning-driven Many-objective Memetic Algorithm for Solving the Workflow Scheduling Problem in Cloud Environment

With increasing complex workflow application and computational resources requirement, distributed computing has attracted growing attention. Meanwhile, cloud computing has emerged as a prominent solution due to its elasticity, heterogeneity, and on-demand capabilities. However, data security and execution reliability in cloud are still urgent issues that need to be addressed. Based on the data encryption and task redundancy mechanism, this paper presented a many-objective workflow scheduling problem (RSWSP) with the objectives of minimizing the execution time, cost, risk, and non-reliability. Then, a two-stage learning-driven many-objective memetic algorithm (TMMA) with tailored designs is introduced to address the RSWSP. First, several problem-specific heuristics are employed for cooperative initialization, generating a diverse set of initial solutions. Second, a two-stage global diversification approach is implemented to explore the problem space, which clusters the population into sub-populations and adoptive selects leader solutions based on the state of the population. In addition, a learning-driven local intensification strategy is incorporated for exploitation, encompassing six neighbor search operators and a Q-learning-based selection mechanism. Extensive experiments have been conducted to validate the performance of TMMA. The statistical comparison reveals that the TMMA is superior to state-of-the-art algorithms in solving the RSWSP in terms of solution quality and robustness.