Latency-Constrained Dynamic Computation Offloading in Mobile Edge Computing using Multi-Agent Reinforcement Learning

Mobile edge computing (MEC) facilitates the development of compute-intensive and real-time applications on mobile devices by providing computing resources in proximity of users. To take full advantage of MEC, making optimal offloading decisions is critical. In this paper, we study the computation offloading of latency-constrained tasks from multiple users to an edge server under a stochastic environment with time-varying wireless channels and dynamically variable set of active users. To lower the mutual interference experienced by users while accessing a set of shared channels, we utilize game theory to formulate the decision-making process of users as a general-sum Markov game model. Then, we provide a mathematical proof to demonstrate the equivalency of the proposed game model to a weighed potential game, which guarantees the presence of at least one pure-strategy Nash Equilibrium (NE) point due to the finite improvement property. Next, we design multi-user computation offloading algorithms using a NE-based multi-agent reinforcement learning (MARL) technique to achieve the equilibrium solution of the game in a decentralized manner. Numerical results show that the proposed algorithms can effectively improve the convergence rate and greatly reduce the system-wide energy cost, outperforming the previously studied learning-based multi-agent algorithms.