Joint Optimization of Packet Scheduling and Energy Harvesting for Energy Conservation in D2D Networks: A Decentralized DRL Approach

This study investigates the optimization of proportional fair (PF) and energy efficiency in simultaneous wireless information and power transfer (SWIPT)-based device-to-device (D2D) networks considering the residual battery levels of D2D users to increase the network lifetime. We establish an optimization model that determines the subchannel allocation and transmission power levels for D2D users, to maximize an objective function that combines user fairness and energy efficiency. To tackle this problem in a distributed manner, we propose a multi-agent deep reinforcement learning (DRL) model. Given that fairness considerations necessitate information about other agents, we employ the long short-term memory (LSTM) algorithm to estimate the parameters of other D2D pairs within the state space of the multi-agent DRL model. Through simulations, we compare the performance of our proposed algorithm with that of existing iterative algorithms, namely, exhaustive search (ES) and gradient search (GS). The results demonstrate that the proposed multi-agent DRL approach achieves a solution that is nearly globally optimal, while maintaining a lower computational complexity due to the parallel computing of multi-agent DRL. Furthermore, the proposed algorithm reduces the standard deviation of residual battery levels among D2D pairs and contributes to an increased network lifetime.