Multi-Stage Volt/VAR Support in Distribution Grids: Risk-Aware Scheduling With Real-Time Reinforcement Learning Control

The ever-increasing penetration of intermittent renewable resources in low-voltage power grids necessitates efficient operational strategies for voltage regulation as well as power scheduling of the available resources. In this paper, a risk-aware Volt/VAR support framework followed by a real-time reinforcement learning controller is presented for three-phase distribution systems. In the risk-aware stochastic scheduling stage, the legacy voltage regulating assets along with inverter-based photovoltaics (PVs) and energy storage system (ESS) are optimized considering day-ahead and intra-day markets. Moreover, demand response (DR) and voltage reduction plans are included in the proposed scheduling framework. By incorporating voltage-dependent load modeling in this study, the implementation of the voltage reduction plan reduces energy consumption in feeders by running the network at lower permissible voltage limits. The shiftable loads under the DR program are employed for peak shaving and to reduce operational costs. The result shows that DR implementation also reduces dependencies on the operation of traditional devices. The stochasticity of abrupt changes in PV generations is represented as the Gaussian Mixture Model (GMM), indicating a non-unimodal probability distribution in day-ahead PV forecasting errors. The scenario sets for uncertain variables are then reduced using a fuzzy clustering technique. Decisions made in the scheduling, associated with PV inverters and ESS operation, are revised with a real-time controller, i.e., Deep Deterministic Policy Gradient (DDPG) reinforcement learning. The DDPG is adopted in the control stage of the framework considering the detailed modeling of unbalanced three-phase distribution grids to minimize the voltage deviation and power ramping of ESS. The performance of the proposed multi-stage scheme is verified using a three-phase active distribution grid under different scenarios.