mmWave Beam Selection in Dynamic Multi-Path Environments: A POMDP Approach

mmWave systems are integral parts of 5G+ wireless systems. Large bandwidths allocated at above 20GHz translate to large data rates otherwise unattainable at lower frequencies. However, signals in the mmWave bands require highly directional beams to overcome strong attenuation and do not propagate through objects along Line-of-Sight (LoS) paths. In this work, we study scenarios with direct LoS and reflected Non-Line-of-Sight (NLoS) paths, where the LoS paths are blocked temporarily. The so-called beam selection problem aims to choose beams to establish communication between two mmWave enabled devices and determine how long the communication with the chosen beam should last. Considering the system's state, defined as the LoS blockage, is observable in one choice of the beams but not others, we formulate the problem as a generalized case of Partially Observable Markov Decision Process (POMDP). The resulting policies result in the maximization of the reward (throughput) of the system, which are demonstrated through numerical examples.