Traffic Offloading and Power Allocation for Green HetNets Using Reinforcement Learning Method

In order to satisfy the boosting mobile traffic demand, the deployment of small cells has been regarded as a feasible solution. But the growth of network infrastructure leads to a tremendous increase of energy consumption. Using renewable energy harvested from the environment to power the small cell base station, forming green heterogeneous networks (HetNets), can help reduce the conventional energy consumption. However, the stochastic user demand and random renewable energy harvesting amount have brought new challenges for the network operation. Based on reinforcement learning, this paper proposes a decentralized and a centralized base station operation scheme. Assuming each base station operates individually, the energy efficiency maximization problem is modeled as a general-sum game. After defining the state, action and reward of each base station, the problem can be solved by multi-agent reinforcement learning. Assuming there is a centralized controller, then the whole network can be seen as a huge agent, thus, the problem can be solved using deep reinforcement learning since the state space is too complicated. Simulation results show the centralized scheme shows higher performance but need more signaling overheads. The energy efficiency is lower for the decentralized scheme but it can be realized easier in real life. Both our proposed scheme can achieve significant energy efficiency improvement compared to the greedy scheme.