Pre-training and Fine-Tuning Based Meta-Reinforcement-Learning for Grid-Forming-Based Energy Storage System Operation to Recover Power Grid Frequency

With the increasing penetration of renewable energy, the frequency control of modern power systems faces significant challenges due to the intermittent feature of renewable energy and the reduced inertia of connected power electronic devices. The grid-forming-inverter-based energy storage can potentially resolve above challenges as it behaves as a controllable voltage source behind a coupling reactance, which is much like a synchronous generator. This paper applies a novel meta deep reinforcement learning method, model-agnostic meta-learning (MAML), on real-time and adaptive controller parameter adjustment of grid-forming-based energy storage systems. This approach focuses on adjusting several of the critical controller parameters of the grid-forming inverters, and thus could provide fast and adaptive active power to recover the grid frequency in different disturbances and operation conditions of the power grids. At the training stage, a meta-policy is trained with various disturbance and operation condition data, and then the pre-trained meta-policy can quickly and effectively adapt to new disturbances and operation conditions using only a small amount of data and a few training iterations. Simulation results on IEEE 39-bus system with high penetration of renewable energy demonstrate that the MAML outperforms other deep reinforcement algorithm in frequency control across unseen scenarios.