DRL Based Beam Management for Joint Sensing and Communications in HSR mmWave Wireless Networks

The abundant available spectrum resources have made millimeter wave (mmWave) communications the key feature of the fifth generation (5G) mobile communications, allowing ultra-high transmission capacity. Additionally, mmWave bands, already widely used in radar systems, show a great advantage in environment sensing. Based on these observations, to satisfy the ever-growing mobile service requirements, and meanwhile to improve the maintenance efficiency for high-speed railways (HSRs), in this paper, we present the joint sensing and communication HSR mmWave wireless network, where two mmWave beams are intelligently controlled to provide broadband communications and environment sensing, respectively. Moreover, to mitigate the inter-beam interference between communication beams and sensing beams, we propose a deep reinforcement learning (DRL) based beam management scheme, where the beamwidth and inter-beam spacing are adaptively adjusted according to dynamic wireless environments. Simulation results demonstrate that our proposed scheme can better balance the communication capacity and the sensing performance compared to conventional schemes with fixed beamwidth and inter-beam spacing.