Delay-Aware Two-Time-Scale Scheduling for mmWave Systems With Mobility and Environment Knowledge

A two-time-scale approximate Markov decision process (MDP) is proposed to optimize the uplink queuing performance of a millimeter wave (mmWave) communication system. Exploiting the wireless sensing techniques, the locations of signal reflectors, blockers, and mobile agents, as well as the mobility pattern of agents, become available knowledge to facilitate predictive scheduling. Notice that the state variation of the wireless channel fading and transmission queue is much faster than that of mobile agents’ locations. The joint optimization of uplink power adaptation, time allocation, and analog beamforming of all the frames is formulated as a two-time-scale MDP with the average uplink energy, queuing length, and buffer overflow rate in the minimization objective. The scheduling in the larger time scale with an infinite horizon, i.e., the analog transceiver beamforming, adapts with the random motion of mobile agents; whereas the scheduling in the smaller time scale with a finite horizon, i.e., uplink power and time allocation, adapts with both the small-scale channel fading, queue dynamics and large-scale random motion. The optimal solution of two-time-scale MDP requests iterative optimization between Bellman’s equations of both time scales, whose computation complexity is prohibitive. A novel low-complexity solution framework is then proposed to obtain the optimal larger-time-scale and sub-optimal smaller-time-scale policies, where the performance of the proposed scheme is bounded analytically. Benefiting from the motion sensing and blockage prediction, the proposed scheduling scheme outperforms existing benchmarks in the numerical simulations.