Deep Reinforcement Learning for UAV-Assisted Spectrum Sharing: A Minority Game Approach

This paper investigates the dynamic spectrum sharing scheme in an unmanned aerial vehicle (UAV)-assisted cognitive radio network. A UAV-assisted device-to-device (D2D) secondary network shares the licensed spectrum band with an air-ground primary link, where the primary transmitter (PT)’s spectrum occupancy status is dynamically changing over time and space. Each secondary transmitter (ST) makes independent spectrum access decision in a fully decentralized manner, based on the automatic repeat request (ARQ) signals received from the primary receiver (PR) and its own secondary receiver (SR) in the previous time slots. We model the distributed spectrum access problem among non-cooperative STs as a minority game, where each ST optimizes its spectrum access policy to maximize the expectation of its cumulative rate while controlling the aggregate interference to the PR. Furthermore, we extend to address the fairness issue by considering an additional fairness constraint across the SU agents. To deal with the unknown and dynamic environment, we propose an independent deep Q-network (IDQN) based decentralized spectrum access algorithm for solving the equilibrium among the STs. Simulation results demonstrate that the STs can adaptively adjust their spectrum access strategies according to the spectrum occupancy states of the PT without explicit coordination.