Energy Efficient Traffic Offloading in Multi-Tier Heterogeneous 5G Networks Using Intuitive Online Reinforcement Learning

The energy efficiency in multi-tier heterogeneous 5G networks is a critical issue due to the fact that the macro base stations (BSs) power consumption is considerably high and proportional to their traffic load. Traffic offloading from macrocells to small cells is envisioned as a potential solution to improve energy efficiency in 5G heterogeneous networks. However, traffic offloading causes traffic congestion at small cells, interference as a result of small cells transmissions, and aggregate power consumption of small cells. These factors make the traffic offloading procedure more challenging. In this paper, we propose a conditional traffic offloading scheme, which relies on macrocells and small cells system load information to determine the most energy efficient traffic offloading strategy, select the proper operation mode of small cells, and fulfill macro users applications' quality of service (QoS) requirements. The proposed scheme is developed using a novel intuitive online reinforcement learning methodology to perform the conditional traffic offloading in which each macro BS conjectures the offloading strategies of other macrocells. The convergence of the proposed scheme is proved and numerical results demonstrate its capability to achieve fast convergence with QoS guarantee and significant energy efficiency.