Deep Reinforcement Learning Guided Framework for State Estimation of Li-ion Batteries: A Control-Oriented Approach With High Adaptability

An accurate and robust state estimation of Lithium-ion batteries (LIBs) is crucial for safe and reliable operations. However, due to the modeling uncertainties and complex nonlinear behavior of LIBs in practical scenarios, existing methods fell short of the desired accuracy and adaptability. This article anchors a proportional integral derivative (PID) observer with deep reinforcement learning (DRL) to develop a battery state estimation framework with enhanced performance, stability, and adaptability. First, the PID observer is constructed using a second-order battery equivalent circuit model, and an actor-critic DRL agent is then integrated with the designed observer to learn the optimal policy for observer control. A dynamic reward function with adaptive constraints is formulated to avoid unproductive explorations and aid efficient learning and convergence. In addition, an augmented model bias is also introduced into the battery model to compensate for the model inaccuracies in real time. After successful training, the trained DRL agent, based on the learned optimal policy, auto-adjusts the PID control according to operating conditions, making the proposed method highly adaptive. Finally, a test bench is designed to verify its improved accuracy, enhanced robustness against uncertainties, faster convergence, and comparative usefulness. Moreover, the high adaptability against real electric vehicle data highlights the potential of the proposed framework in improving the safety of LIBs in practical applications.