Optimal Resource Allocation for GAA Users in Spectrum Access System Using Q-Learning Algorithm

Spectrum access system (SAS) is a three-tier layered spectrum sharing architecture proposed by the Federal Communications Commission (FCC) for Citizens Broadband Radio Service (CBRS) 3.5 GHz band. The available 150 MHz spectrum is dynamically shared among Incumbent Access (IA), Primary Access Licensees (PAL) and General Authorized Access (GAA) users. IA users are the highest priority federal military users, PAL users are the licensed users and the GAA users are the least priority unlicensed users. In this scenario, PAL operators are willing to give access to their idle spectrum to GAA users to generate extra revenue. SAS will ensure to protect IA users and PAL users from interference caused by lower-tier users. It is the responsibility of SAS to allocate resources to GAA users but the method to do so is left open. In this article, a novel auction algorithm based on Q-learning for dynamic spectrum access (SAS-QLA) is proposed. In SAS-QLA, multiple GAA users dynamically and intelligently bid using Q-learning to access PAL reserved idle channels. SAS will decide to allocate the channels to GAA users with maximum bidding offers. GAA users have their own quality of service (QoS) demands i.e., transmission rate, packet loss, bidding efficiency, and maintain the preference of available PAL reserved idle channels based on Q-learning considering the available QoS. The proposed scenario is also modeled as a knapsack NP-hard problem and solved using dynamic programming and distributed relaxation method. Numerical results demonstrate the effectiveness of the SAS-QLA algorithm in improving the bidding efficiency, maximizing the data rate per unit cost and spectrum utilization.