A Multi-Step Joint Q-learning Cooperative Algorithm for Regional Interconnected Power Systems

Distributed renewable energy, due to its own randomness, intermittency and unpredictability, will lead to frequency fluctuations in the power system when it is connected to the grid on a large scale, posing a serious challenge to the security, stability and economy of the interconnected grid. This paper outlines a systematic strategy aimed at mitigating the adverse effects of strong random disturbances and frequency instability associated with the integration of new energy sources into the power grid. The approach centers on optimizing the Automatic Generation Control (AGC) within a distributed multi-region multi-energy interconnection power system. Consider battery storage and conventional units, collaboratively involved in AGC frequency regulation, a novel time-difference algorithm DQ(σ, λ) is proposed, and the introduction of hybrid sampling factors unifies the Sarsa algorithm with the Expected-Sarsa algorithm to achieve the optimal solution of the optimal strategy, and then realize the optimal cooperative in distributed multi-region. The real grid environment is simulated by the constructed two-region interconnected power system model with energy storage and the distributed multi-region multi-energy synergy power system model in order to verify the practical engineering effectiveness of the DQ(σ, λ) algorithm. The results show that the DQ(σ, λ) algorithm has fast convergence characteristics, and is able to solve the frequency instability problem and obtain the optimal synergy among the regions of the power grid.