Safe Deep Reinforcement Learning-Based Real-Time Operation Strategy in Unbalanced Distribution System

Distribution systems have become more complex because of the increasing penetration of renewable energy resources. Moreover, unbalanced voltages are one of the voltage quality issues in distribution systems. Conventional optimization methods are time-consuming because of solving each scenario. Deep reinforcement learning (DRL) is an effective technique for real-time operations to overcome the time-consumption problem. This paper proposes a safe deep reinforcement learning (SDRL)-based distribution system operation method to mitigate unbalanced voltage. The SDRL method incorporates a learning module (LM) and a constraint module (CM). The deep deterministic policy gradient (DDPG) algorithm is introduced in the LM. The CM is designed with quadratic programming (QP) to satisfy control constraints. The proposed SDRL model controls the energy storage system (ESS) in the distribution system to improve voltage balancing. A modified IEEE-13 node test feeder is applied to validate the performance of the proposed method. Finally, a conventional heuristic optimization method, particle swarm optimization (PSO), is simulated to compare with the proposed method in terms of performance and time consumption.