Multiagent Primal-Dual DDPG-Based Reactive Power Optimization of Active Distribution Networks via Graph Reinforcement Learning

The large-scale integration of distributed energy resources into active distribution networks (ADNs) may significantly intensify voltage fluctuations and increase network losses. Traditional model-based reactive power optimization (RPO) approaches depend on existence of accurate system models. On the other hand, conventional reinforcement learning (RL) methods largely ignore the spatial characteristics of the ADNs during training, allowing agents to have an inadequate perception of the system state. To address these challenges, this article proposes a multiagent deep RL approach that integrates graph learning with RL for the learning of RPO strategies in ADNs. Specifically, the active distribution network is divided into multiple regions, with each region being controlled by an agent. The agents collaborate to achieve the global RPO goal. The perception capability of the agents is enhanced by adopting graph attention networks during the feature extraction phase. In the training phase, a primal-dual method is employed to manage constraints effectively. During the execution phase, each agent controls the photovoltaic inverters, electric springs, and capacitor banks based on the strategies developed in the training phase. The performance of the proposed approach is validated by a series of experiments on the IEEE-33 system, along with comparisons versus some existing data-driven deep RL methods.