DRL-Based Computation Rate Maximization for Wireless Powered Multi-AP Edge Computing

In the ongoing 5G and upcoming 6G eras, the intelligent Internet of Things (IoT) network will take increasingly important responsibility for industrial production, daily life and so on. The IoT devices with limited battery size and computing ability cannot meet many applications brought out by the data-driven artificial intelligence technique. The combination of wireless power transfer (WPT) and edge computing is regarded as an effective solution to this dilemma. IoT devices can collect radio frequency energy provided by hybrid access points (HAPs) to process data locally or offload data to the edge servers of HAPs. However, how to efficiently make offloading decisions and allocate resource is challenging, especially for the networks with multiple HAPs. In this paper, we consider the sum computation rate maximization problem for a WPT empowered IoT network with multiple HAPs and IoT devices. The problem is formulated as a mixed-integer nonlinear programming problem. To solve this problem efficiently, we decompose it into a top-problem of optimizing offloading decisions and a sub-problem of optimizing time allocation under the given offloading decisions. We propose a deep reinforcement learning (DRL) based algorithm to output the near-optimal offloading decision and design an efficient algorithm based on Lagrangian duality method to obtain the consequent optimal time allocation. Simulations verified that the proposed DRL-based algorithm can achieve more than 95 percent of the maximal computation rate with low complexity. Compared with the common actor-critic algorithm, the proposed algorithm has the substantial advantage in convergence speed, achieved computation rate and running time.