Centralized Deep Reinforcement Learning Method for Dynamic Multi-Vehicle Pickup and Delivery Problem With Crowdshippers

Crowdshipping problem can be challenging as the platform are continuously but sporadically receiving crowdshippers and delivery tasks with heterogeneous origin and destination. In this paper, the dynamic multi-vehicle pickup and delivery problem with crowdshippers (DMV-PDPC) is considered. Leveraging the deep reinforcement learning framework, the attention model with centralized vehicle network (AMCVN) method is developed. Unlike traditional heuristic or existing vehicle-changing methods, AMCVN integrates a centralized vehicle network (CVN) that can observe the state information of all vehicles, enhancing its overall performance. In each decision-making step, the CVN monitors the state of the vehicles and selects one of the vehicles. Subsequently, the attention-based route generating network (RGN) determines the next node to be visited by the chosen vehicle. Instead of using a penalty term in the reward function to regulate the sequence of visits to pickup and delivery nodes, a more precise control method, namely the rolling mask scheme (RMS), is implemented. The method’s evaluation is carried out via a simulation experiment using a real-world road network. This evaluation demonstrates that the proposed method effectively tackles the DMV-PDPC challenge, outperforming current state-of-the-art learning-based models and heuristic methods. Moreover, the method shows exceptional generalization capabilities, as evidenced by its adaptability to different numbers of tasks and vehicles.