Improved Decentralized Fuzzy Q-learning for Interference Reduction in Heterogeneous LTE-Networks

Recently, the concept of Self Organizing Networks (SON) has received significant attention in the context of Heterogeneous Networks (HetNets) to overcome indoor coverage problems as well as to improve the efficiency of current macrocell systems. Nevertheless, the detrimental factor in such networks is the co-channel interference between macrocells and small cells (femto, pico, relay), as well as among neighboring small cells which can dramatically decrease the overall capacity of the network. In order to cope with network densification and heterogeneity, self-configuring and self-optimizing strategies are of utmost importance to boost network capacity and control interference among different tiers. In this paper, we focus on decentralized cross-layer interference mitigation techniques in a Orthogonal Frequency Division Multiple Access (OFDMA) based HetNet deployment, whereby small cells autonomously optimize their downlink transmissions. We propose a Reinforcement Learning (RL) framework, based on an improved decentralized Q-learning algorithm for small cells sharing the macrocell spectrum. Since the major drawback of Q-learning is its slow convergence, we propose an improved effective initialization procedure. The proposed algorithm will be compared with a basic Q-learning algorithm, with Fuzzy Q-learning and basic approaches from literature. Interestingly, our proposal protects especially the so-called 5-th% cell-edge macro User Equipments (UEs) and is very close to the case that no small cells are activated as interferers in the system by at the same time enhancing the small cell performance.