A Deep-Reinforcement-Learning-Based Beam Prediction Scheme for Vision-Aided mmWave Wireless Communications

Millimeter wave (mmWave) wireless communications are significant technologies that support Internet of Things (IoT) systems to achieve fast and stable data transmission, and the guarantee of its communication quality usually depends on accurate beam prediction. Due to the advantage of not relying on channel state information, beam prediction schemes using visual data and artificial intelligence become popular. Most of the current vision-aided beam prediction schemes directly predict the index of the optimal beam in the codebook. However, these methods are only applicable to certain codebooks and have limited generalization and scalability. To address the issues, we propose a vision-aided beam prediction scheme based on deep reinforcement learning (DRL). The scheme takes the original image as input and extracts the position and velocity of the user through the object detection algorithm. Subsequently, combined with the current state of the base station, it outputs a continuous angle value and finally matches it with the beam index in the codebook. Furthermore, we integrate the attention mechanism into the actor network of the deep deterministic policy gradient (DDPG) and propose a scheme of DDPG with attention mechanism (DDPG-A), which can perform differential processing on features, thereby enhancing algorithmic performance. The simulation test utilizing real datasets demonstrates that the proposed scheme has good generalization and scalability while considerably reducing the beam training overheads.