Rapid Solution for Flexible Pickup and Delivery Services Problem Based on Improved Actor-Critic Deep Reinforcement Learning

The problem of the Flexible Pickup and Delivery Services Problem (FPDSP) arises from the actual needs of multi-warehouse management strategies and is one of the key challenges in the current urban distribution logistics industry. The problem aims to quickly calculate the route planning in complex scenarios to ensure that the total traveling time of the vehicle is minimized while meeting the time window requirements. To address this problem, we propose a deep reinforcement learning method based on the Actor-Critic algorithm to quickly calculate the approximate optimal solution of FPDSP. Specifically, we propose a Transformer Model with Parallel Encoders (TMPE). The model efficiently extracts order features through parallel encoders and then uses serial decoders to optimize the fusion of feature information to optimize the order selection process. In addition, we designed a reward function to reduce the number of repeated pickups made by the vehicle at the same consignor’s location between different orders, thereby effectively reducing the vehicle’s total travel time. Experimental results show that our method can quickly find feasible solutions to the problem compared with heuristic methods on seven different datasets. At the same time, compared with all baseline methods, the number of optimal solutions of our method reaches 14, which significantly improves the problem-solving ability. This result provides a new solution for optimizing pickup and delivery logistics in multiple warehouses in cities in the future.