Hierarchical Multi-Agent Deep Reinforcement Learning for Multi-Objective Dispatching in Smart Grid

Due to the expansion of new energy sources, the complexity and difficulty of power grid dispatching are further increased, especially simultaneously considering security, economic and environmental factors. Existing power grid dispatching methods, such as the bisection method and proportional control, are not competent for the multi-objective complex dispatching. This paper proposes a hierarchical multi-object deep deterministic policy gradient (HMO-DDPG) algorithm to dispatch the smart grid with new energy sources. In this algorithm, the lower Decision Layer uses multi-agent architecture to make decisions for every generator unit, while the upper Optimization Layer evaluates the security, economic and environmental factors of the decisions in the whole power grid. Besides, a double evaluation mechanism deployed in the two layers is proposed to make the decision more reasonable and comprehensive. Moreover, a topology analysis method is used to avoid the islanding problem during smart grid dispatching. Experiments carried out on the power grid simulator provided by China Electric Power Research Institute show HMO-DDPG can enable the power grid to operate safely for over 80% days of a year without exceeding the thermal stability limit of all branches. Moreover, the economic cost is 1.80% less, and the utilization rate of new energy is 70.48% higher than that of the traditional dispatching methods on average.