Customized Energy Management for Fuel Cell Electric Vehicle Based on Deep Reinforcement Learning-Model Predictive Control Self-Regulation Framework

Deep reinforcement learning (DRL) has been widely used in the field of automotive energy management. However, DRL is computationally inefficient and less robust, making it difficult to be applied to practical systems. In this article, a customized energy management strategy based on the deep reinforcement learning-model predictive control (DRL-MPC) self-regulation framework is proposed for fuel cell electric vehicles. The soft actor critic (SAC) algorithm is used to train the energy management strategy offline, which minimizes system comprehensive consumption and lifetime degradation. The trained SAC policy outputs the sequence of fuel cell actions at different states in the prediction horizon as the initial value of the nonlinear MPC solution. Under the MPC framework, iterative computation is carried out for nonlinear optimization problems to optimize action sequences based on SAC policy. In addition, the vehicle's usual operation dataset is collected to customize the update package for further improvement of the energy management effect. The DRL-MPC can optimize the SAC policy action at the state boundary to reduce system lifetime degradation. The proposed strategy also shows better optimization robustness than SAC strategy under different vehicle loads. Moreover, after the update package application, the total cost is reduced by 5.93% compared with SAC strategy, which has better optimization under comprehensive condition with different vehicle loads.