Demand Side Management and Peer-to-Peer Energy Trading for Industrial Users Using Two-Level Multi-Agent Reinforcement Learning

Over the past few years, with more concerns on energy equity, energy security and environmental sustainability, peer-to-peer (P2P) energy trading market has increasingly attracted attention to energy users. With the rising electricity price and the growing public doubt to unsustainable development, more efforts are given to the integration between P2P energy trading and industrial practice. To date, the current research work has emphasized electricity cost reduction using demand-side management (DSM) and P2P energy trading market to promote energy sharing and create a win–win situation for all industrial market participants. However, less attention has been given to evaluating industrial production. With this attention, this paper formulates a cost minimization problem for multiple discrete manufacturers, where DSM and P2P energy trading market are integrated to optimize manufacturing productivity and energy consumption. Moreover, this paper proposes a two-level reinforcement learning (RL) method to obtain the optimal DSM and energy trading strategies for discrete manufacturers. Unlike the existing RL approaches, the proposed method depicts a two-level learning approach, where the upper-level program suggests a production plan for the manufacturer to improve the cost return, while the lower-level program optimizes the net benefits. Further, to verify the performance of the proposed method, simulations are conducted based on four different case studies, including different weather and load conditions. Numerical results demonstrate that the proposed algorithm provides a better production plan with enhanced learning speed and economic benefits compared to the existing RL approaches.