Dual-Path Beam Tracking for Service Continuity of Ultra-Reliable and Low-Latency Communications

Multi-antenna millimeter-wave (mmWave) communication systems have become a promising approach to improve throughput. However, mmWave narrow beams are susceptible to blockages, making it difficult to guarantee continuous services for ultra-reliable and low-latency communications (URLLC). To address this issue, we propose a novel dual-path beam tracking framework and develop a Recurrent Neural Network-based Constrained Deep Reinforcement Learning (RCDRL) algorithm to optimize the beam search sets of the beam tracking algorithm. The objective is to minimize the total time and frequency resources allocated for beam sweeping, beam tracking, and data transmission, subject to the constraint on the service interruption probability of URLLC. A pre-training method is developed to improve the initial performance and stability of the RCDRL algorithm. Comprehensive evaluation results indicate that the proposed approach outperforms other baseline methods in terms of the tradeoff between service interruption probability and resource utilization efficiency. Specifically, the RCDRL algorithm reduces the service interruption probability by three orders of magnitude compared with a standardized single-beam tracking method at the cost of sacrificing the resource utilization efficiency by 14.6%. In addition, our policy achieves lower service interruption probability and higher resource utilization efficiency compared with an existing dual-beam tracking algorithm.