Distributed interference management using Q-Learning in cognitive femtocell networks: New USRP-based implementation

Femtocell networks have become a promising solution in supporting high data rates for 5G systems, where cell densification is performed using the small femtocells. However, femtocell networks have many challenges. One of the major challenges of femtocell networks is the interference management problem, where deployment of femtocells in the range of macro-cells may degrade the performance of the macrocell. In this paper, we develop a new platform for studying interference management in distributed femtocell networks using reinforcement learning approach. We design a complete MAC protocol to perform distributed power allocation using Q-Learning algorithm, where both independent and cooperative learning approaches are applied across network nodes. The objective of the Q-Learning algorithms is to maximize aggregate femtocells capacity, while maintaining the QoS for the Macrocell users. Furthermore, we present the realization of the algorithms using GNURadio and USRP platforms. Performance evaluation are conducted in terms of macrocell capacity convergence to a target capacity and improvement of aggregate femtocells capacity.