A DDPG-based Transfer Learning Optimization Framework for User Association and Power Control in HetNet

Deep Reinforcement Learning (DRL) has been a useful technique for achieving resource management in the heterogeneous network (HetNet), such as user association and power control (UAPC). In this paper, we apply the DRL-based method to investigate the joint UAPC problem with the objective of improving the sum rate and reducing the transmission time. However, the change of HetNet settings always degrades the performance of a pre-trained model and training a new model from scratch is costly. To generate resource management algorithm for such dynamically changing scenarios, we propose the Deep Deterministic Policy Gradient Algorithm (DDPG)-based Transfer Learning Algorithm (DTL), which can train a new DRL agent for the joint UAPC problem of new HetNet effectively and quickly. Finally, in terms of the sum rate and transmission time of users, simulation results demonstrate the proposed method can achieve a near-optimal performance with significant training speedup.