Smoothing Tie-line Power for Microgrids by Controlling District Cooling System Based on Soft Actor-Critic Reinforcement Learning

Renewable energies (RENs) bring significant power fluctuations to tie-line power in microgrids, which may cause high operational risk and deteriorate power quality. As a large-scale flexible load, district cooling system (DCS) is an ideal resource to suppress short-time tie-line power fluctuations through load control. However, the real-time control for DCS is quite challenging due to the following two reasons: i) DCS is commonly a complex thermal system that is hard to be modeled accurately, so that most model-based control methods cannot work for it; ii) Uncertainties from the REN’s power fluctuations make it difficult to achieve zero-steady control results. To address the above issues, this paper proposes a model-free reinforcement learning method for DCS, which can obtain an adaptive control strategy to smooth tie-line power. First, the DCS control problem is described as a sequential decision-making problem and modeled as a Markov decision process (MDP). Then, the soft actor-critic algorithm is adopted to solve the optimal control strategy in the designed MDP, which does not need prior knowledge of the DCS model. The implemented experiments verify the effectiveness of the proposed method.