DQN-Based Predictive Spectrum Handoff via Hybrid Priority Queuing Model

In cognitive radio networks (CRNs), spectrum handoff techniques help the interrupted secondary user (SU) vacate the licensed channel and seek for another suitable channel to resume its unfinished transmission. However, multiple interruptions from primary users and various latency requirements impose enormous challenges to spectrum handoffs. To this end, we propose a new hybrid priority queuing model for predictive spectrum handoffs and derive the closed-form expression for the extended data delivery time (latency performance), and then a deep Q-network (DQN)-based algorithm is designed to minimize the transmission latency for SUs. Furthermore, the transfer learning method is also introduced in our spectrum handoff algorithm to accelerate the learning process in which a newly added SU can obtain the initial loss function from its nearest neighbor. Simulation results show that the proposed spectrum handoff method outperforms the conventional approaches based on reinforcement learning in terms of the latency performance.