Real-Time Energy Management for Plug-in Hybrid Electric Vehicles via Incorporating Double-Delay Q-Learning and Model Prediction Control

Plug-in hybrid electric vehicles (PHEVs) have been validated as a preferable solution to transportation due to its great advantages in fuel economy promotion, harmful emission reduction and mileage anxiety mitigation. While, designing an effective energy management strategy to allocate the power between battery and engine is critical to improve the performance of powertrain in PHEVs. To this end, a real-time energy management strategy is proposed via incorporating double-delay Q-Learning and model predictive control (MPC). First, the energy management for PHEV is transformed into a nonlinear optimal control problem, and the vehicle speed predictor based on convolutional neural network is proposed to forecast vehicle speed in MPC. Then, based on the predicted vehicle speed, the double-delay Q-Learning algorithm is implemented to solve the receding horizon optimal problem in the MPC module. The simulation is conducted to verify the performance of the proposed strategy, and the results showcase that incorporating the double-delay Q-Learning into MPC can effectively improve the adaptability of energy management to dynamic environment, and meanwhile achieve a similar fuel consumption of the offline stochastic dynamic programming-based strategy. In addition, the single-step computation time of the proposed strategy is less than 23 milliseconds, highlighting its significant potential in online implementation.