Hierarchically correlated equilibrium Q-learning for multi-area decentralized collaborative reactive power optimization

A hierarchically correlated equilibrium Q-learning (HCEQ) algorithm for reactive power optimization that considers carbon emission on the grid-side as an optimization objective, is proposed here. Based on the multi-area decentralized collaborative framework, the controllable variables in each region are divided into several optimization layers, which is an effective method for solving the limitations posed by dimensionality. The HCEQ provides constant information on the interaction between the state-action value function matrices, as well as on the cooperative game equilibrium among agents in each region. After acquiring the optimal value function matrix in the pre-learning process, HCEQ is able to quickly achieve an optimal solution online. Simulation of the IEEE 57-bus system is performed, which demonstrates that the proposed algorithm can effectively solve multi-area decentralized collaborative reactive power optimization, with the desired global search capabilities and convergence speed.