Intelligent Beam Management for Millimeter-Wave Cellular Networks in IoT System

In wireless distributed systems represented by the Internet of Things(IoT), the millimetre-wave (mmWave) communication with ample bandwidth and immunity to interference has been considered as an essential technology. However, beam management becomes complicated in such a dense network with the rapid growth of small-cell base stations (SCBSs). In this letter, we propose a reinforcement learning (RL) based beam management scheme, where a multi-agent deep deterministic policy gradient (MADDPG) is applied on each SCBS to maximize the long-term system throughput while guaranteeing the quality of service. The MADDPG algorithm learns from the experience of dynamic variations in the network environment caused by the mobility of IoT devices. Hence, the beam management in the SCBS is optimized according to the reward or penalty when severing different devices. Numerical results reveal the convergence performance of the MADDPG and its superiority in improving the system throughput compared with other typical RL algorithms and the traditional beam management method.