Deep Deterministic Policy Gradient Based Computation Offloading in Wireless-Powered MEC Networks

With the upsurge of Internet of Things (IoT), mobile edge computing (MEC) and wireless power transfer (WPT) are becoming promising methods to relieve IoT devices from the limited computation capacity and battery capacity. In this paper, we study the joint optimization of the local computing central processing unit (CPU) frequency and the offloading ratio in a dynamic WPTMEC network. In the proposed network, all the IoT devices follow a harvest-then-compute protocol so as to be charged by the MEC server and then compute the tasks. Our goal is to minimize the long-term averaged maximal computing delay with stochastic task arrivals and time-varying wireless channels. It is challenging to find the optimal solution due to the uncertain network information and the non-convex objective function. To tackle this problem, we propose a deep reinforcement learning-based deep deterministic policy gradient (DDPG) framework, which can learn a centralized policy to coordinate the local computing CPU frequencies and offloading ratios of IoT devices. Simulation results show that the proposed DDPG algorithm can achieve up to 73.2% and 16.8% gain in terms of the averaged maximal computing delay compared to the local-computing-only method and the deep Q-network algorithm, respectively.