DRL-Assisted Energy Minimization for NOMA-Based Dynamic Multi-User Multi-Access MEC Networks

In this article, a dynamic multi-user multi-access mobile edge computing (MEC) network where users split their tasks into multiple parts and offload them concurrently to multiple MEC servers via NOMA is investigated. In order to reduce the long-term energy, a complicated time-dependent non-convex problem with many deeply-coupled variables is formulated. We employ a novel approach that integrates the theoretical derivation and the deep reinforcement learning (DRL) algorithm to solve the problem in real time. In particular, we decompose the considered energy minimization problem into a computational resource and transmission power allocation subproblem as the offloading decision and offloading time of each user are known, and an offloading decision and offloading time allocation subproblem. Then, the first subproblem is decomposed into many independent problems, and we theoretically solve them in parallel and derive the optimal solution. For the second subproblem, in order to obtain the real-time allocation of offloading time and offloading decision, we apply a DRL algorithm that adjusts the reward function with a penalty mechanism while the total offloading ratio constraint is unsatisfied or the transmission power and computational resource allocation subproblem does not have a solution. Via simulations, we verify the optimality of the proposed theoretical derivation based solutions, and, demonstrate that the system performance can be significantly improved by the proposed approach.