Multi-Agent Reinforcement Learning for Providing Flexibility Services in Local Energy Communities

Highlighting the need to evaluate power system flexibility associated with incorporating variable renewable resources, there is a shift towards leveraging demand-side flexibility. By promoting the collaborative association of consumers and distributed energy resources (DERs) in the form of local energy communities (LECs), flexibility service provision is entrusted to community managers. This paper proposes a data-driven framework suitable for bidding strategy in LECs to enable energy trading and providing flexibility services. Employing a model-free decision-making approach through deep Q network (DQN) methods based on the coordination of multiple agents combines the advantages of manager-centric (centralized) architecture and a data-driven approach. The proposed deep reinforcement learning (DRL) algorithm optimizes the community manager’s bidding in local energy and flexibility markets simultaneously. Moreover, dissatisfaction and operation cost functions are presented to increase flexible user engagement incentives. Results of test cases prove that designing a LEC where flexibility bids are integrated with day-head (DA) energy scheduling mitigates the imposed agent’s costs.