Deep Reinforcement Learning-Based Prosumer Aggregation Bidding Strategy in a Hierarchical Local Electricity Market

This paper investigates the application of deep reinforcement learning (DRL) algorithm for the decision-support of a prosumer aggregation in a hierarchical local electricity market (LEM) comprising a peer-to-peer (P2P) market and a corrective market. The agent first submits bids/asks to the P2P market where prosumer aggregations are able to trade electricity directly with each other. After that, the agent participates in the corrective market, where the market operator formulates the corrective market as an AC optimal power flow (OPF) problem to ensure the system is operated within its operational limits. A DRL algorithm, namely Twin Delayed Deep Deterministic Policy Gradient (TD3), is used to find the strategic bidding strategy. The algorithm is tested on a real medium-voltage distribution grid to evaluate the effectiveness of the strategic bidding method. The result of the case study demonstrates that the agent can derive trading strategies to obtain high profits based on the TD3 algorithm.