Deep Reinforcement Learning based Handoff Algorithm for Finite Buffer Traffic in HetNets with End-to-End Network Slices

With the continuous emergence of new applications, network slicing function has been regarded as a promising technology to provide flexible services. When network slicing function is enabled in heterogeneous networks, the handoff decision becomes challenging due to the large number of base stations (BSs) and Network Slices (NSs). Considering the latency of the real applications, the NS handoffproblem when finite buffer traffic model is adopted in HetNets with end-to-end NSs is studied in this paper. The Markov decision process of the NS handoff problem is modeled to optimize the cumulative service profit, handoff cost and delay penalty. The status of core NSs, radio access NSs and task flows are considered in the state. The handoff algorithm based on Double DQN is proposed, and an illegal action filter is added to the output layer of the Q-current network to speed up convergence. Simulation results show that the proposed algorithm has better overall performance than other traditional algorithms evaluated in this paper. Besides, it can adapt to different system design emphases.