A Distributed Dynamic Inertia-Droop Control Strategy Based on Multi-Agent Deep Reinforcement Learning for Multiple Paralleled VSGs

The virtual synchronous generator (VSG) control method for energy storage is a promising way to improve the frequency stability of the power system with large-scale renewable energy. However, due to the mismatch parameters, power oscillations may occur when multiple VSGs are connected in parallel to the grid. In this paper, the relationship between oscillation and the inertia-droop parameters distribution of multiple paralleled VSGs is derived by the simplified frequency response model. Furthermore, to coordinate the inertia-droop parameters of paralleled VSGs for oscillation damping, this paper formulates the parameter tuning problem as a Markov game with an unknown transition function and proposes a distributed dynamic inertia-droop control strategy based on multi-agent deep reinforcement learning (MADRL). Based on the local and adjacent VSG information, each agent learns the optimal inertia-droop control parameters independently by interacting with the environment. The soft-actor-critic (SAC) framework is introduced to train each agent. The well-trained agent can modify the parameters of VSG dynamically to suppress the power oscillation under different operating conditions. Finally, several time-domain results demonstrate the effectiveness and robustness of the proposed approach.