Action Advising and Energy Management Strategy Optimization of Hybrid Electric Vehicle Agent Based on Uncertainty Analysis

The energy management control strategy of hybrid electric vehicles (HEVs) is a critical research field. During driving, a vehicle encounters various uncertain and even intricate driving cycles for a vehicle. Although reinforcement learning has clear advantages in energy management control strategies, fuel consumption remains high in the driving cycles. To solve these problems, this study proposes an uncertainty-based reinforcement learning method, which uses a distributed uncertainty function to select the action when faced with intricate regions of driving cycles. The algorithm implementation processes are as follows: First, uncertainty is calculated by the uncertainty module; then, the action values, in situations of low uncertainty, are derived from the original reinforcement learning algorithm, while in situations of high uncertainty, they are obtained from the uncertainty module. Results indicate that through the application of the uncertainty-based reinforcement learning method, the fuel economy is improved by about 5% compared to the deep deterministic policy gradient (DDPG) algorithm and about 11% higher than the deep Q-network (DQN) algorithm under intricate driving cycles.