Online Learning for Position-Aided Millimeter Wave Beam Training

Accurate beam alignment is essential for the beam-based millimeter wave communications. The conventional beam sweeping solutions often have large overhead, which is unacceptable for mobile applications, such as a vehicle to everything. The learning-based solutions that leverage the sensor data (e.g., position) to identify the good beam directions are one approach to reduce the overhead. Most existing solutions, though, are supervised learning, where the training data are collected beforehand. In this paper, we use a multi-armed bandit framework to develop the online learning algorithms for beam pair selection and refinement. The beam pair selection algorithm learns coarse beam directions in some predefined beam codebook, e.g., in discrete angles, separated by the 3 dB beamwidths. The beam refinement fine-tunes the identified directions to match the peak of the power angular spectrum at that position. The beam pair selection uses the upper confidence bound with a newly proposed risk-aware feature, while the beam refinement uses a modified optimistic optimization algorithm. The proposed algorithms learn to recommend the good beam pairs quickly. When using $16\times 16$ arrays at both transmitter and receiver, it can achieve, on average, 1-dB gain over the exhaustive search (over $271\times 271$ beam pairs) on the unrefined codebook within 100 time steps with a training budget of only 30 beam pairs.