Performance Study in HetNets Using Dual Connectivity and Deep Reinforcement Learning

The commercialization of 5th generation (5G) technology began in several countries in 2020. However, it takes time to transition to a 5G network. 5G brings new techniques such as millimeter-wave (mm-wave), IoT, network densification, slicing, etc. Therefore, users with 4th generation (4G) or 5G capabilities must be able to seamlessly adjust to the existing dual infrastructure by implementing the new 5G radio and current 4G-Long-Term Evolution (LTE) technology. Dual connectivity (DC) is a solution proposed by the 3rd Generation Partnership Project (3GPP) in mobile heterogeneous networks (HetNets) to make this type of communication possible. Moreover, DC is a solution to frequent handovers (HO). Compared to conventional networks, user equipment (UE) in an ultra-dense mm-wave cellular network suffers HO events more frequently, leading to longer service interruption times and performance deterioration due to blockages. Machine Learning (ML) is one of the recent solutions used in mobile networks to solve several problems. Deep Reinforcement Learning (DRL) is a branch of machine learning that combines deep learning with reinforcement learning. Here, we aimed to improve the network performance using Proximal Policy Optimization (PPO) and Advantage Actor-Critic (A2C), two policy-based DRL algorithms, to examine our model and discuss the most performant algorithm among them. The results indicate that in most of the applied instances, PPO is frequently the algorithm that performs the best.