Provable Stability of Reinforcement Learning-Based Frequency Control in Low-Inertia Power Systems

Modern converter-dominated power grids require fast and adaptive frequency control while guaranteeing system stability and safety. However, existing reinforcement learning (RL) approaches for inverter-based control often sacrifice theoretical guarantees for adaptivity, making them unsuitable for safety-critical grid operations. In this paper, we propose a physics-guided control decomposition that decouples the stability and adaptation mechanisms within a provably stable RL framework. The controller comprises two components: (i) a strongly monotonic nominal control which is used as a baseline that guarantees exponential stability via Lyapunov energy decay, and (ii) a sign-controlled residual that enables robust adaptation to unknown and time-varying inertia and damping. We derive explicit design margins that certify local exponential stability under actuator saturation and parameter variations. The resulting method maintains linear computational complexity and is suitable for real-time deployment on grid-forming inverters. Simulation results on the IEEE 39-bus system show that our method achieves faster settling time, reduced frequency overshoot, and improved robustness to parameter drift compared to safe RL baselines such as Lyapunov RL techniques. As a result, this work provides a scalable and theoretically grounded framework for learning-based frequency control in low-inertia power systems.