Multi-Agent Deep Reinforcement Learning for D2D-assisted MEC system with Energy Harvesting

It is challenging for delay-sensitive task offloading in a Device-to-Device(D2D) assisted MEC system with energy harvesting devices due to the dynamic load level at each edge node and the fluctuation of harvested energy. In this paper, we investigate the joint task assignment and frequency control problem for MEC system integrated with D2D communication and energy harvesting. D2D link is leveraged to reduce the traffic load of the cellular base station and improve the utilization of computing resources in MEC. The main objective is to minimize the long-term average task delay with battery energy constraint. We first formulate a large scale non-linear integer programming problem that makes task assignment and CPU frequency decisions for each active device at each time slot. To solve the sequential decision problem with huge multi-dimensional discrete action space, we propose a multi-agent reinforcement learning algorithm based on Multi-Agent Proximal Policy Optimization (MAPPO) technique. Our algorithm can work efficiently under non-stationary environment and convergence fast. Experimental results show that our proposed algorithm outperforms other baseline algorithms.