Deep Reinforcement Learning for Cost-Efficient Power Control in Multi-Cell Networks

Energy consumption is a critical concern in multi-cell networks. To reduce the energy cost, renewable energy has been accommodated to offer cellular networks with auxiliary energy supply. Most existing studies of cellular networks powered by hybrid energy sources are based on ideal battery models. However, the energy storage efficiency of batteries in reality is influenced by environmental factors and the natural degradation of batteries over time. In this paper, we investigate the cost-efficient transmit power control in multi-cell networks with hybrid energy sources and time-varying energy storage efficiency. Considering such dynamic networks characterized by the stochastic property of energy harvesting, channel conditions, and time-varying energy storage efficiency, deep reinforcement learning (DRL) is adopted to learn the optimal power control strategy without requiring prior knowledge of the environment. To enhance the robustness and accommodate varying levels of exploration and exploitation during different training phases of DRL, an enhanced proximal policy optimization algorithm is proposed. Specifically, a dynamic clipping threshold is designed based on an adaptive adjustment mechanism, which allows modifying the clipping threshold in accordance with the agent's performance. Simulation results show that our proposed DRL algorithm can significantly improve the cost efficiency of the multi-cell network compared with other DRL algorithms.