Distributed Actor-Critic Neural Networks-Based Structure for Frequency Synchronization in Isolated AC Microgrids

Unlike conventional model-based distributed control of the isolated AC microgrid (MG) in which the stability of the closed-loop system can not be guaranteed especially when the operation condition is deviated from its nominal value, by exploring real-time measurement data, a model-free reinforcement learning (RL) design for frequency synchronization will be studied in this paper. First, the original frequency synchronization problem will be re-formulated as a distributed pinning-based consensus problem. According to the optimal control theory, a model-based on-policy RL will be developed. To further distinguish the target policy for performance evaluation with the behavior policy for gathering experiences from real-time measured data, a model-free off-policy RL control will also be exploited for effective parallel implementations. Moreover, to enhance online computation ability, the actor-critic neural networks structure is also exploited to approximate model-free off-policy RL control. Real-time simulation studies are performed on real-time simulations OPAL-RT to validate the superior dynamic performance of the proposed distributed model-free method for autonomous frequency synchronization in the isolated AC MG.