An Algorithm Combining Deep Reinforcement Learning and Dynamic Programming for Solving TSP-D

Unmanned aerial vehicles serve as versatile terminal devices applicable across various scenarios, involving the coordination among humans, drones, and the environment. This integration represents a typical application of the fusion development of ‘human-machine-environment’. The outstanding performance exhibited by UAVs enables their application in traditional truck delivery systems. However, effectively integrating truck and drone route planning poses a new challenge, known as the Traveling Salesman Problem with Drone (TSP-D). The current algorithm only focuses on TSP-D instances where the number of truck nodes is smaller than or equal to the number of drone nodes. Therefore, we decompose the TSP-D problem into two parts: path point planning and path generation, proposing a two-stage solution algorithm that combines deep reinforcement learning and dynamic programming. Firstly, by designing a novel reward function, we employ reinforcement learning algorithms to train pointer networks to generate the sequence of path points for trucks and drones. Subsequently, dynamic programming is used to collaboratively plan accurate paths for trucks and drones by coordinating their path points. This paper demonstrated the feasibility of the algorithm in experiments with 11, 21, and 51 nodes. The results indicate that as the number of visiting nodes increases, the drone’s flight distance gradually exceeds the truck’s driving distance. This further confirms that the drone can effectively become the main force in the task, significantly reducing the operational time of TSP-D.