Auxiliary Network Enhanced Hierarchical Graph Reinforcement Learning for Vehicle Repositioning

Affected by people’s dynamic social activities, the imbalance between vehicle supply and demand in the Mobility-On-Demand(MOD) system is a common phenomenon. To improve traffic efficiency, an Auxiliary Network Enhanced Hierarchical Graph Reinforcement Learning (AHGRL) method is proposed for vehicle repositioning. Firstly, a hierarchical graph reinforcement learning (HGRL) framework is designed. The complex vehicle repositioning problem in real road networks is divided into many sub-tasks and multiple reinforcement learning algorithms are designed to solve decision problems of different levels. Traffic congestion is also considered and road nodes are clustered dynamically. And then an auxiliary graph reinforcement learning (AGRL) algorithm is designed for the actuator. It contains the prediction branch and the repositioning branch. States and rewards of agents could be designed accurately with the support of the prediction branch. The two branches cooperate in an auxiliary way to achieve excellent forecasting and repositioning effects. Finally, to enable efficient multi-vehicle coordination, a discrete Soft Actor-Critic algorithm is adopted in the repositioning branch, which learns multiple optimal actions for vehicles in the same area. Comparative experiments with real data demonstrate the effectiveness of our method. And ablation experiments verify the effectiveness and universality of the HGRL framework and the AGRL algorithm.