User Association in Dense Millimeter Wave Networks With Multi-Channel Access Points Using the Whittle Index

In dense millimeter wave (mmWave) networks, user association, i.e., the task of selecting the access point (AP) that each arriving user should join, significantly impacts the network performance. We consider a dense mmWave network in which each AP has multiple channels and can simultaneously serve different users using different channels. The different channels of an AP are susceptible to both blockage, which is common to all the channels of an AP, and frequency-selective fading, which is, in general, different for different channels. In each time slot, a user arrives with some probability. Our objective is to design a user association scheme for selecting the AP that each arriving user should join, so as to minimize the long-term total average holding cost incurred within the system, and thereby achieve low average delays experienced by users. This problem is an instance of the restless multi-armed bandit problem, and is provably hard to solve. We prove that the problem is Whittle indexable and present a method for calculating the Whittle indices corresponding to the different APs by solving linear systems of equations. We propose a user association policy under which, when a user arrives, it associates with the AP that has the lowest Whittle index in that time slot. Our extensive simulation results demonstrate that our proposed Whittle index-based policy outperforms user association policies proposed in prior research in terms of the average delay, average cost, as well as Jain’s fairness index (JFI).