Green transmit power assignment for cognitive radio networks by applying multi-agent Q-learning approach

As the scarce spectrum resource is becoming over-crowded, cognitive wireless mesh networks (CogMesh) indicate great flexibility to improve the spectrum utilization by opportunistically accessing the authorized frequency bands. In this paper, we consider non-cooperative green power assignment in CogMesh with the consideration of energy efficiency. The problem is modeled as a stochastic learning process. We extend the single-agent Q-learning to a multi-user context, and propose a conjecture based multi-agent Q-learning scheme to obtain the optimal strategies with private and incomplete information. A learning secondary user performs Q-function updates based on the conjecture about other secondary users' behaviors. Simulations are used to verify the performance of our algorithm and demonstrate its effectiveness of improving the energy efficiency.