Reinforcement Learning-based Smart Inverter Control with Polar Action Space in Power Distribution Systems

To tackle the challenge of voltage regulation under high solar photovoltaics (PV) penetration, the slow timescale control of conventional voltage regulating devices can be combined with fast timescale control of smart inverters. In this paper, we develop a two-timescale Volt-VAR control (VVC) framework. The slow time-scale control of voltage regulating devices is achieved by a model-based approach. The fast timescale control of smart inverters is attained with a reinforcement learning-based method. The deep deterministic policy gradient (DDPG) algorithm is adopted to control the setpoints of both real and reactive power of smart inverters. The control policy of smart inverters is learned from the historical operational data without relying on accurate distribution network secondary circuit parameters. Simulation results on the IEEE 34-bus feeder show that the proposed framework can determine near optimal set points of smart inverters in real-time operations. Compared with existing reinforcement learning based smart inverter control, our approach achieves lower line losses, voltage deviations, and active power curtailment.