Decentralized Q-Learning for Aggregated Interference Control in Completely and Partially Observable Cognitive Radio Networks

This paper deals with the problem of aggregated interference generated by multiple cognitive radios (CR) at the receivers of primary (licensed) users. In particular, we consider a secondary CR system based on the IEEE 802.22 standard for wireless regional area networks (WRAN), and we model it as a multi-agent system where the multiple agents are the different secondary base stations in charge of controlling the secondary cells. We propose a form of real-time multi-agent reinforcement learning, known as decentralized Q-learning, to manage the aggregated interference generated by multiple WRAN cells. We consider both situations of complete and partial information about the environment. By directly interacting with the surrounding environment in a distributed fashion, the multi-agent system is able to learn, in the first case, an optimal policy to solve the problem and, in the second case, a reasonably good suboptimal policy. Simulation results reveal that the proposed approach is able to fulfill the primary users interference constraints, without introducing signaling overhead in the system.