A Soft Actor-Critic Approach for Power System Fast Frequency Response

As the integration of renewable energy sources (RES) is increasing in the electric grid, the level of inertia is decreasing which makes frequency stability in the power grid more challenging. To address this issue, this paper proposes a reinforcement learning (RL)-based approach for power system fast frequency response (FFR). The proposed method uses a neural network controller based on soft actor-critic (SAC) to learn an optimal control policy. The SAC RL-based FFR is trained using a reduced order power system frequency dynamics model in Simulink. To analyze the effectiveness of the proposed method, it is compared with another FFR approach, Model Predictive Control (MPC). The results show that the proposed model-free method can efficiently provide FFR in power systems and outperforms the model-based MPC with almost 24% more reduction in frequency deviation and almost 5 times faster computation time.