Multi-Objective and Multi-Agent Deep Reinforcement Learning for Real-Time Decentralized Volt/VAR Control of Distribution Networks Considering PV Inverter Lifetime

PV inverters can provide prompt and flexible reactive power support to voltage/var control (VVC) of distribution networks, but their lifetime can be significantly reduced due to additional reactive power output. To balance the conflict between the VVC performance and the inverter lifetime, this paper firstly proposes a multi-objective real-time decentralized VVC framework. Then, a multi-objective multi-agent deep reinforcement learning (MOMADRL) algorithm is developed to coordinate the PV inverters through centralized training and decentralized implementation, offering an advantageous alternative to traditional model-based methods and eliminating the need for centralized communication. By incorporating multiple actors and an actors-based parallel training scheme (ABPTS), multiple policies can be concurrently learned to find the Pareto front. Lastly, sensitivity analysis is conducted to obtain the Pareto front and provide guidelines for operators in choosing the proper weighting factors. Simulation results on a 141-bus distribution system validate the effectiveness and efficacy of the proposed method, as voltage regulation can be achieved with an extended inverter lifetime, as well as reduced network energy loss. The proposed MOMADRL algorithm is also applicable to other multi-objective problems as a universal multi-objective data-driven approach.