Improved DQN Path Planning Method Based on Transformer

Aiming at the problem that the slow convergence of Deep Q-Network (DQN) in complex path planning tasks, a DQN method based on Transformer (T-DQN) is proposed. To enhance the model's ability to capture long-range dependencies, the linear layers in DQN are replaced with a multi-head attention mechanism. The self-attention mechanism of Transformer can dynamically weight historical state information, which enables the agent to learn global path features more efficiently in complex environments. In addition, to optimize training sample utilization, accelerate key experience learning, and further improve convergence efficiency, a priority sampling strategy is employed. Finally, simulations are conducted on both simple and complex map path planning to validate the effectiveness and superiority of the proposed model. Simulation results show that the proposed T-DQN algorithm achieves faster convergence compared to the DQN algorithm.