Leveraging AI for Enhanced Power Systems Control: An Introductory Study of Model-Free DRL Approaches

The power grids nowadays are facing increasing complexity and uncertainty due to the continuously growing penetration of renewable energy sources, such as photovoltaic (PV) and wind power, as well as the emerging uncertain and dynamic nature of demand-side factors such as Electric Vehicle, portable energy storage, etc. To effectively manage these challenges, artificial intelligence (AI) technologies, particularly model-free deep reinforcement learning (DRL), have risen as a powerful tool for power system control. This paper presents an in-depth review of the state-of-the-art applications of model-free DRL in power systems control. The review is focused on various model-free DRL approaches utilized in addressing uncertainty caused by stochastic factors from renewable generations, demand-side dynamics, and power system contingencies. Specifically, it investigates how model-free DRL techniques are employed to facilitate decision-making in the frequency control, voltage control, and optimal power flow control in the grid. The benefits, challenges, and limitations of these technologies are revealed, shedding light on recent advancements in the field and showcasing the performance of these methods across diverse power system scenarios. By synthesizing the findings from extensive research, this paper also highlights the limitations, key future research directions, and recommendations for deploying model-free DRL in power system control.