Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading

Peer-to-peer (P2P) energy trading system has the ability to completely revolutionize the current household energy system by sharing energy among residents. As the number of customers employing distributed energy resources (DERs) such as solar rooftops increase, innovation in the double auction market (DA) system is becoming more significant. In this paper, a novel model-based, multi-agent asynchronous advantage actor-centralized-critic with communication (MB-A3C3) approach is carried out. Previous studies are limited since they suffer from unpredictable behavior in renewable energy resources and a large number of prosumers in the peer-to-peer market. As for the model-based strategy, we forecast the trading price and trading quantity in the daily energy trading system in order to overcome unpredictable issues. For the large number of prosumers, the multi-agent and multithreading RL has been chosen as our backbone since the prosumers’ behavior can be diverse; time-series clustering is introduced based on their daily trading behavior. With its environmental model and multi-threaded mechanism, MB-A3C3 is seen to be most efficient in carrying out tasks regards time and precision. The model is conducted on a large scale real-world hourly 2012–2013 dataset of 300 households in Sydney having rooftop solar systems installed in New South Wales (NSW), Australia. Results reveal that the MB-A3C3 approach outperforms other reinforcement learning methods (MADDPG and A3C3), producing lower community energy bills for 300 households. When internal trade (trading among houses) increased and external trade (trading to the grid) decreased, our multiple agent RL (MB-A3C3) significantly lowered energy bills by 17%. In closing the gap between the real-world and theoretical problems, the algorithms herein aid in reducing customers’ electricity bills.