Neural Pontryagin Optimal Controller for Lossy Energy Storage with Nonlinear Efficiency

Solving real-world lossy battery control problems are challenging tasks, as the complex charging dynamics involving nonlinear efficiency are usually unrevealed to the system operators. It is thus challenging and time consuming to find the optimal control actions numerically. To deal with both modeling and computation challenges, in this paper, we integrate Neural Networks with the Pontryagin’s Maximum Principle (PMP), and propose a sample efficient framework Neural-PMP. The resulting controller can be implemented for systems with unknown and complex dynamics, and is readily available for multi-battery, multi-timestep energy storage arbitrage task. By taking an iterative approach, the proposed framework not only utilizes the accurate surrogate models parameterized by neural networks, it also efficiently recovers the optimality conditions along with the optimal action sequences via PMP conditions. Numerical simulations on grid-connected lossy battery control tasks demonstrate our proposed Neural-PMP is a general and versatile computation tool for finding optimal solutions. And compared to the widely applied model-free and model-based reinforcement learning (RL) algorithms, our Neural-PMP achieves higher sample efficiency and performance in terms of control objectives. We also open-source our Neural-PMP implementations for achieving more competitive energy storage control¹.