Distributed Secondary Control Strategy Based on $Q$-learning and Pinning Control for Droop-controlled Microgrids

A distributed secondary control (DSC) strategy that combines
$Q$
-learning and pinning control is originally proposed to achieve a fully optimal DSC for droop-controlled microgrids (MGs). It takes advantages of cross-fusion of the two algorithms to realize the high efficiency and self-adaptive control in MGs. It has the following advantages. Firstly, it adopts the advantages of reinforcement learning in autonomous learning control and intelligent decision-making, driving the action value of pinning control for feedback adaptive correction. Secondly, only a small part of points selected as pinned points needs to be controlled and pre-learned, hence the actual control problem is transformed into a synchronous tracking problem and the installation number of controllers is further reduced. Thirdly, the pinning matrix can be modified to adapt to plug-and-play operation under the distributed control architecture. Finally, the effectiveness and versatility of the proposed strategy are demonstrated with a typical droop-controlled MG model.