Optimal Dynamic Task Scheduling in Heterogeneous Cloud Computing Environment

Cloud computing (CC), often necessitates dynamic adjustments due to its inherently fluid nature. In this paper, we introduce a novel dynamic task scheduling model that incorporates reward and holding cost considerations, leveraging the Continuous-Time Markov Decision Process (CTMDP) framework in heterogeneous CC systems. The primary goal of this model is to maximize the overall system reward for the Cloud Service Provider. By solving the Bellman Optimality Equation using the value-iteration method, we can derive an optimal scheduling policy for the dynamic task scheduling model. Additionally, to enhance its practicality in real-world scenarios, we incorporate a model-free reinforcement learning algorithm to obtain the optimal policy for our proposed model without requiring explicit knowledge of the system environment. Simulation results demonstrate that our proposed model outperforms two common static scheduling methods.