Knowledge-Augmented Population-Based Deep Reinforcement Learning for Real-Time Network-Constrained Economic Dispatch of Large-Scale Power Grid

Deep reinforcement learning (DRL) for real-time network-constrained economic dispatch (NCED) has witnessed a surge of interest in recent years, for its advantages in online look-ahead decision-making and adaptability for uncertainties. However, regular DRL methods struggle to adapt to large-scale grid environments, due to their inherent limitations in computational efficiency and parallelizability. To fill these gaps, this paper proposes a novel knowledge-augmented population-based deep reinforcement learning (PDRL) method for large-scale real-time NCED problems. PDRL perturbs the agent's parameters to generate a population for exploration, and aggregates all individuals' results to construct a surrogate gradient for updates. Such a population-based parameter space exploration paradigm enables efficient exploration capability and high parallelizability. Moreover, leveraging the physical knowledge of the power grid, a novel sequential safe projection (S2P) technique was developed, which significantly enhances agent decision safety and alleviates training difficulty, allowing for better adaptation to the large-scale power grid. Numerical simulations on the 39-bus, 500-bus and 2383-bus systems demonstrate that compared with the state-of-the-art (SOTA) DRL methods, the proposed method shows better solution optimality, decision security, computational performance as well as the adaptability to large-scale power grids.