Deep Reinforcement Learning for Solving the Vehicle Routing Problem in Practical Logistics

The vehicle routing problem with simultaneous pickup-delivery and time windows (VRPPDTW) is applicable to a wide range of practical scenarios within the domains of transportation and logistics. When addressing this complex problem, conventional heuristic algorithms have exhibited limitations in effectiveness, primarily due to the need for expert knowledge in heuristic design, which often prioritizes immediate improvements and leads to convergence towards local optima. To overcome this challenge, we propose a deep Q-network-based iterated local search algorithm (DQN-ILS). This algorithm employs a deep Q-learning network to automatically learn a policy for selecting heuristics to improve the current solution. Simultaneously, an adaptive perturbation strategy is designed to avoid becoming trapped in local optima. In this way, our algorithm can better balance the trade-off between exploration and exploitation when solving VRPPDTW. Experimental studies conducted on a real test suite of large-scale VRPPDTW instances validate the superiority of the proposed DQN-ILS over several recently proposed methods.