Power Allocation in HetNets with Hybrid Energy Supply Using Actor-Critic Reinforcement Learning

In order to utilize renewable energy and save conventional energy, energy saving traffic offloading in heterogeneous networks (HetNets) has been attracting attentions by many literature in recent years. This paper focuses on the power allocation problem after user scheduling or traffic offloading, with the goal of optimizing the network energy efficiency. Due to the stochastic property of wireless channels and green energy situation, meanwhile the state transition probabilities and expected rewards for all states are unknown in mobile environments, a stochastic optimal power allocation policy for in HetNets should be learned. We formulate and solve the energy-efficiency oriented power allocation problem in HetNets with the policy-gradient-based actor-critic algorithm, which is a model-free reinforcement learning framework and has been successfully used in applications, such as robotics and operations research. We use the parameterized policy to select actions and update the parameters using the gradient ascend method. The numerical simulations are shown to demonstrate the performance of the proposed algorithm.