Parallel-Reinforcement-Learning-Based Online Energy Management Strategy for Energy Storage Traction Substations in Electrified Railroad

The traction power supply system (TPSS) is the only source of power for electric locomotives. The huge power fluctuations and complex operating conditions of the TPSS pose a challenge to the efficient operation of energy storage traction substations. The existing energy management strategies are difficult to achieve accurate charging and discharging, difficult to modify the control rules in real time, and have poor migration capability. For comparison, the reinforcement learning (RL) algorithms can address the shortcomings of rule-based energy management strategies due to their model-free feature. Therefore, this article proposes an energy management strategy based on parallel reinforcement learning (PRL) to improve the efficiency of energy utilization while speeding up the convergence of the algorithm. More specifically, a Markov decision framework is established for capturing the energy management process. The Monte Carlo sampling process is also improved to achieve offline optimization by PRL algorithms and reduce the impact of low-value power fragments on iteration speed. Meanwhile, the algorithm is modified to enable online updates. The case study shows that compared with other energy management strategies, the PRL-based energy management strategy has faster convergence speed, higher energy exchange efficiency, and better migration capability, and can adapt to various complex working conditions.