Uncertainty-driven Active Reinforcement Learning for Energy Management Strategy in Electrified Vehicles

Reinforcement learning (RL), as a data-driven optimization method, enables electrified vehicles to train optimal energy management strategy (EMS) in complex and dynamic environments. However, traditional RL methods often face inefficiency during the exploration process, particularly in high-dimensional continuous state spaces, leading to slow convergence or a tendency to fall into local optima. To address these issues, this paper proposes an uncertainty-driven active RL (UaRL) for EMS, which is built upon the soft actor-critic algorithm, using the dual Q-values difference as a measure of uncertainty. When the uncertainty increases significantly, the active learning mechanism is activated, allowing the UaRL system to select more reliable actions through rule-based methods and guide the improvement of the RL policy. Experimental results show that the novel UaRL approach has significant advantages in energy efficiency, convergence speed, and policy stability. Compared to vanilla RL, the proposed method improves economic performance by an average of 14.68%. It also demonstrates superior performance in battery health management and capacity stability. This study provides an efficient and reliable hybrid learning framework for EMS in electrified vehicles.