Flexible Power Load Control Based on Hierarchical Multi-Agent Reinforcement Learning

The characteristics of high randomness and low controllability of new energy power generation have gradually increased the pressure on the supply and demand balance of the power grid and continued to rise in costs. Therefore, the stable control of building energy systems is considered to be an increasingly severe challenge. Using advanced control strategies to control building energy storage can significantly improve the building's energy flexibility, allowing the system to operate more stably. Reinforcement learning (RL) has been shown to be effective in smoothing out peaks in system power consumption. However, when energy management is expanded from a single building to a building cluster, uncoordinated control strategies for each building may lead to excessive peaks in the building cluster, negatively affecting the stability of the power grid. To address these challenges, this paper proposes a coordinated energy management method based on hierarchical multi-agent reinforcement learning (HRL). Compared with traditional reinforcement learning (RL), HRL introduces a hierarchical structure so that each agent can learn information from other agents and can handle more complex and large-scale problems. The controller designed in this paper aims to optimize the energy consumption of each building while smoothing the overall load curve of the building complex. Experimental results show that compared with traditional rule-based manual optimization controllers, the HRL controller performs well in coordinating energy management, not only improving energy utilization efficiency but also enhancing the stability and reliability of the power grid.