An Energy Management Strategy Based on DDPG With Improved Exploration for Battery/Supercapacitor Hybrid Electric Vehicle

Given that transportation contributes to 23% of global energy-related greenhouse gas emissions, the electrification of the transport sector is an inevitable trend. This paper presents a hybrid energy management method that utilizes the Deep Deterministic Policy Gradient (DDPG) algorithm. To enhance its exploration capability and balance exploration and exploitation, the Locality Sensitive Hashing (LSH) algorithm and Prioritized Experience Replay (PER) algorithm are introduced. DDPG, being a deterministic strategy, typically explores the environment by adding noise to the action space, which lacks effective exploration capability. Therefore, the LSH algorithm is employed for approximate counting of state variables in high-dimensional continuous state spaces. This encourages agents to explore less frequently occurring state variables, thereby improving the algorithm’s exploration ability. Additionally, the integration of the PER algorithm enables more efficient utilization of limited sample data during experience replay, leading to improved learning efficiency and stability. In addition, periodic updates are used to compensate for the additional computational consumption caused by counting. Simulation results on driving cycles from the typical road conditions demonstrate that the proposed algorithm has stronger exploration ability and better management performance comparing with EMSs based on DDPG, DQN and Q-Learning. The LSH-PER-DDPG based EMS achieves a significant 8.5% reduction in lithium battery power consumption, a 6.25% increase in supercapacitor recovery, and a substantial 15.7% reduction in instantaneous fluctuation of lithium battery output current.