Multimodal Fusion Assisted Mmwave Beam Training in Dual-Model Networks

Millimeter wave (mmWave) communication adopts narrow beams to combat severe propagation loss, resulting in unbearable training duration and overhead. To address this problem, this article proposes a beam training scheme based on a multimodal fusion network. We investigate the feature fusion process of mmWave channel power leakage (CPL) and sub-6 GHz channel state information (CSI) to provide richer feature information. Compared with the uni-modal counterparts (mmWave only or sub-6 GHz only), the complementarity and interactivity between different modalities make the proposed network more accurate and robust. Specifically, we propose a multimodal fusion network based on the instantaneous signal received by the mmWave wide beam and sub-6 GHz CSI to estimate the optimal narrow beam. In addition, the additional mmWave narrow beam decided by the predicted probabilities is employed to calibrate beam direction further. Moreover, two optimization strategies are presented based on the proposed multimodal fusion network. Firstly, to improve the robustness to noise in wide beam training, the long and short memory (LSTM) network is employed to track the motion direction of the user and calibrate the mmWave beam. And the wide beam for training is selected according to the previous prediction probability to reduce the training overhead. Secondly, since beam training focuses on partially specific information, to focus on its related features, we introduce the self-attention mechanism to improve the global consistency of the network. Numerical results illustrate that the proposed scheme can achieve significantly more accurate results with less overhead than traditional and existing deep learning-based methods.