Deep Reinforcement Learning-Based mmWave Beam Alignment for V2I Communications

Millimeter wave (mmWave) communication can meet the requirements of vehicle-to-infrastructure (V2I) systems, for high throughput and ultra-low latency. However, searching for the optimal beamforming vectors in highly dynamic environments, incurs considerable training overhead. And it is a huge challenge to achieve beam alignment between receivers and transmitters. This paper proposes a beam alignment algorithm based on vehicle position information, to achieve fast beam alignment in the V2I network. In the proposed algorithm, a roadside unit (RSU) obtains a set of candidate beams by the vehicle position information and the double deep Q network (DDQN) algorithm. Then, according to the criterion of maximizing the system spectral efficiency, the optimal beam of the candidate beam set is obtained by the exhaustive search, to achieve fast beam alignment. In this paper, the DeepMIMO dataset is utilized to fully consider the actual scene of V2I, and the effect of Doppler expansion is taken into account in the mathematical model. The simulation results show that the received signal-noise ratio (SNR) of vehicle at different positions is greater than the SNR threshold, which avoids communication interruption and improves the reliability of V2I communications. Meanwhile, we also evaluates the effect of vehicle speed. Compared with other search schemes, the proposed scheme attains higher transmission rates, effectively balances the training overhead and achievable rate, and is suitable for mmWave V2I networks.