Hybrid Decision-Making for Intelligent High-Speed Train Operation: A Boundary Constraint and Pre-Evaluation Reinforcement Learning Approach

Deep Reinforcement Learning (DRL) is the most promising technology for improving high-speed train’s energy efficiency and operation quality. Existing solutions, however, suffer from three significant limitations: 1) They cannot effectively constrain the huge exploration space generated by high-speed trains under high temporal deformability and long-distance trips; 2) The reward function has no adaptability to the different energy-efficiency difficulties of different travel schedules, and the agent will receive incorrect reward signals, requiring manual adjustment; 3) They do not avoid the invalid action sequences of the agent well. To address this challenge, we propose a revolutionary Boundary Constrained and Pre-evaluated Reinforcement Learning (BCPRL) approach to alleviate these issues. This approach combines the Shrink Trajectory Exploration Space (STES) module, the Pre-evaluated Energy-efficiency Scenario Complexity (PESC) module, and the Twin Delayed Deep Deterministic Policy Gradient (TD3) module and uses a hybrid of STES and TD3 to make train operation decision-making to improve the operation quality and learning efficiency of the agent. Numerical experiments validate the effectiveness of the BCPRL approach, which, by drastically reducing the exploration space and getting the agent the correct reward signal, not only maintains excellence in efficiency and punctuality but also far surpasses the other baseline approaches in learning efficiency and robustness.