Deep-Reinforcement-Learning-Based Continuous Workflows Scheduling in Heterogeneous Environments

Workflow scheduling plays a critical role in optimizing completion time and throughput in distributed cloud environments, leveraging the parallelism of heterogeneous computing resources. However, existing workflow scheduling algorithms often fall short due to heuristic limitations and the challenges in adaptability within heterogeneous settings, leading to suboptimal scheduling solutions. In this article, we present a novel deep reinforcement learning (DRL) framework tailored for continuous workflow scheduling in heterogeneous environments. First, we propose an intelligent scheduler that updates the policy network through interactions with a multitenant environment, triggered by scheduling events. Next, the framework incorporates a graph attention network (GAT) and a self-attention multilayer perceptron (MLP) to preprocess the workflow topology and embed dynamic features of ready tasks and available processors into the state input at each scheduling step. Additionally, a k-dimensional tree-based k-nearest neighbors (kNN) algorithm is employed to map the output action vector to a pair of executed ready task and processor, facilitating the transition from continuous to discrete action spaces and addressing challenges associated with dynamic action spaces. Experimental results demonstrate that our method converges effectively in continuous workflow scheduling scenarios and significantly outperforms the best-known methods in terms of average makespan and load balancing efficiency.