Temporal-Difference Learning-Based Stochastic Energy Management for Plug-in Hybrid Electric Buses

Plug-in hybrid electric buses (PHEBs), compared with traditional fuel-driven vehicles, can achieve higher fuel economy and lower pollution emissions. For a PHEB with a single-shaft parallel powertrain, a major challenge for researchers is to find approximate optimal energy management strategies that can run in real time. Motivated by this idea, this paper aims at minimizing PHEB fuel consumption with a temporal-difference (TD) learning method. First, historical driving cycle data from real-world bus routes are collected and processed and parameter variables of TD are introduced. Specially, this process is completed offline. Then, the configuration and main parameters of PHEB are presented, and a control-oriented dynamic system of the PHEB is constructed. Thereafter, the TD learning method based on historical data is introduced. Furthermore, the approximate optimal control strategy for energy management is proposed. Compared with the traditional optimal control strategy, the proposed method can realize real-time running without sacrificing the accuracy of optimization, because the learning method updates the estimates based on other learned estimates without calculating a final outcome. This method can learn directly from the data of running PHEBs without a simplified model of the PHEB, which can avoid the influence of model error. Finally, to verify this method, several different strategies are used for comparison. In addition, experimental results in real-world driving cycles demonstrate that the proposed method can improve the fuel economy obviously by up to 21% compared with a traditional charge-deleting, charge-sustaining scenario. Therefore, this novel method has great potential in realistic applications.