Learning to Optimize User Association and Spectrum Allocation With Partial Observation in mmWave-Enabled UAV Networks

To support large-scale unmanned aerial vehicle (UAV) networks with both payload communication (PC) packets and control and non-payload communication (CNPC) packets, millimeter wave (mmWave) communication is convinced to be a promising solution. However, efficient user association and spectrum allocation are still challenging in mmWave-enabled UAV networks considering that the network state is dynamic and the status information of each UAV is incomplete. In this paper, we investigate a joint UAV association and spectrum allocation problem under a hybrid mmWave sharing paradigm, where PC packets are transmitted over both licensed and pooled bands in a shared manner to achieve high throughput and CNPC packets are transmitted over licensed band in an exclusive manner to guarantee high reliability. To this end, we introduce a strategic form game that can characterize network stochastic states and individual partial observations to reformulate the problem, and then propose a counterfactual regret minimization scheme to achieve its correlated equilibrium (CE). Benefited from the updating mechanism on random
observation-action
pairs, the designed scheme can converge to the corresponding CE solutions for the two types of packets with partial observation. Finally, our simulation results demonstrate the superior performance of the proposed scheme over the baseline schemes.