A reinforcement learning-based routing scheme for cognitive radio ad hoc networks

Cognitive radio (CR) has been proposed to enable unlicensed users (or secondary users, SUs) to exploit the underutilized licensed channels (or white spaces) owned by the licensed users (or primary users, PUs). This article presents a simple and pragmatic reinforcement learning (RL)-based routing scheme called Cognitive Radio Q-routing (CRQ-routing). CRQ-routing is a spectrum-aware scheme that finds least-cost routes taking into account the dynamicity and unpredictability of channel availability and channel quality, as well as interference to PUs. RL is applied to enable each SU node to observe, learn and make action selection that maximizes network performance as time goes by; and this is essential as it may not be feasible to define actions for all possible sets of network conditions. Simulation results show that CRQ-routing minimizes SUs' interference to PUs, SUs' end-to-end delay, SUs' packet loss rate, as well as maximizes SUs' throughput.