Networked Multiagent-Based Safe Reinforcement Learning for Low-Carbon Demand Management in Distribution Networks

This paper proposes a multiagent-based bilevel operation framework for low-carbon demand management in distribution networks considering the carbon emission allowance on the demand side. In the upper level, the load agents optimize the regulation signals for various types of loads to maximize profits; in the lower level, the distribution network operator makes optimal dispatching decisions to minimize the operational costs and calculates the distribution locational marginal price and carbon intensity. The distributed flexible load agent has only incomplete information on the distribution network and cooperates with other agents using networked communication. Finally, the problem is formulated as a networked multiagent constrained Markov decision process, and it is solved using a safe reinforcement learning algorithm called consensus multiagent constrained policy optimization, considering the carbon emission allowance for each agent. Case studies with the IEEE 33-bus, 123-bus and practical 1031-bus distribution network systems demonstrate the effectiveness of the proposed approach in terms of satisfying the carbon emission constraint on the demand side and ensuring the safe operation of the distribution network without complete information exchange.