NoisyNet-DDQN-Based Sequential Handoff Algorithm for UAV Networks with End-to-End Network Slicing

End-to-end network slicing in unmanned aerial vehicle (UAV) networks has attracted significant attention for delivering differentiated services tailored to the quality of service (QoS) requirements of user terminals (UTs). In a sliced UAV network, the dual mobility of both UTs and UAVs, combined with the three-layer association of UT-UAV-network slice (NS), makes handoff a critical and challenging problem for maintaining the communication performance of UTs. This paper proposes a sequential handoff process where a central controller makes handoff decisions for each UT sequentially according to the UT queue. This process is modeled as a Markov decision process (MDP). In the MDP, the state space includes NS available resources, UTs' QoS requirements, and UT queue information. Additionally, the reward function accounts for UTs' QoS and handoff overhead. The double deep Q-network (DDQN) technique is then employed to design a handoff algorithm. This algorithm achieves the optimization goal of ensuring UTs' QoS requirements while avoiding frequent handoffs by maximizing the total reward of the MDP. Additionally, a state-dependent exploration technique with adaptive intensity adjustment, called noisy network (NoisyNet), is utilized to drive the DDQN for effective exploration, thereby enhancing the performance of the handoff algorithm. Finally, the runtime complexity of the proposed NoisyNet-DDQN-based sequential handoff (NoisyNet-DDQN-SH) algorithm is analyzed. The simulation results show that the NoisyNet-DDQN-SH algorithm converges well. Moreover, it exhibits strong robustness. Additionally, the NoisyNet technique effectively enhances the algorithm's performance, making the NoisyNet-DDQN-SH algorithm superior to the vanilla DDQN (with an ϵ-greedy strategy), greedy, maximum signal-to-interference-plus-noise ratio (MaxSINR), and random algorithms.