A Deep Q-Learning Dynamic Spectrum Sharing Experiment

We report results of an experiment in applying deep Q-learning for dynamic spectrum sharing (DSS) in the Alleys of Austin scenario from the DARPA Spectrum Collaboration Challenge. This scenario mimics mobile operations in an urban environment by up to five squads (teams) of soldiers. Each team operates its own wireless network. We consider teamwise– distributed DSS, where there is no central agent to coordinate spectrum usage across teams, but spectrum usage within each team is coordinated by a single member of that team. The spatial distributions of the soldiers creates opportunities for spatial reuse by certain subsets of the teams, and our experiment is set up to evaluate whether the deep Q-learning algorithm can discover and take advantage of these opportunities. The results show that deep Q-learning is able to take advantage of spatial reuse and that doing so results in better performance than a fair-share, disjoint spectrum allocation among the teams.