Stochastic Computation Offloading and Scheduling Based on Mobile Edge Computing

To improve the quality of service (QoS) for mobile users (MUs) and the quality of experience (QoE) of mobile devices (MDs), mobile edge computing (MEC) is a promising approach that offloads a part of the computing task from MDs to nearby MUs. In this paper, we study computation offloading involving multiple users and multiple base stations (BSs), where the MD that is connected to the MU is wirelessly charged and BSs are available to be selected for computation offloading. We model the process of solving an optimal computation offloading policy into a Markov decision process (MDP), in which our goal is to maximize the long-term utility performance. Therefore, a computation offloading policy is obtained based on the energy queue state, the task queue state, and the channel states between the MUs and BSs. To address the problem of high dimensionality in the state space, we decompose the MDP into a series of single-agent MDPs with reduced state spaces and apply an online local learning algorithm to learn the optimal state value functions. Inspired by the structure of the utility function, we propose an algorithm based on combining Q-function reconstruction with the post-decision state. It is proved that the proposed algorithm can converge to an optimal computation offloading policy. The experimental results show that our algorithm achieves significant performance in computation offloading and schedule compared with the other three basic policies.