Reinforcement Learning Based Online Scheduling of Multiple Workflows in Edge Environment

In edge environment, many smart application instances are triggered randomly by resource-constrained Internet of Things (IoT) devices. These application instances usually consist of dependent computation components, which can be modeled as workflows in different shapes and sizes. Due to the limited computing power of IoT devices, a common approach is to schedule partial computation components of multiple workflow instances to the resource-rich edge servers to execute. However, how to schedule the stochastically arrived multiple workflow instances in edge environment with the minimum average completion time is still a challenging issue. To address such an issue, in this paper, we adopt the graph convolution neural network to transform multiple workflow instances with different shapes and sizes into embeddings, and formulate the online multiple workflow scheduling problem as a finite Markov decision process. Furthermore, we propose a policy gradient learning-based online multiple workflow scheduling scheme (PG-OMWS) to optimize the average completion time of all workflow instances. Extensive experiments are conducted on the synthetic workflows with various shapes and sizes. The experimental results demonstrate that the PG-OMWS scheme can effectively schedule the stochastically arrived multiple workflow instances, and achieve the lowest average completion time compared with four baseline algorithms in edge environments with different scales.