Deep Reinforcement Learning-Aided Bidding Strategies for Transactive Energy Market

The concept of transactive energy market (TEM) has been introduced to efficiently balance supply and demand across the electrical networks in a distributed manner. TEM allows consumers to sell/purchase a portion of their excess/lack of energy to/from other local consumers. In this article, a transactive framework is proposed for a distribution network in which transactive agents participate in a local market and submit their bids to local TEM for day-ahead planning. However, due to complexity of the market and effects of a multitude of factors in the market outcome, deriving the optimal bid is not a trivial task. In order to learn the optimal bidding strategy in such a complex system with incomplete information, soft actor-critic method is utilized. Two scenarios are analyzed and compared. The first scenario assumes only one agent uses the learning method, while in the second scenario a number of residents form a coalition with a representative learning agent. Then, allocation of power assigned to the whole coalition between its residents is discussed. Comprehensive simulations are conducted for the 37-bus IEEE distribution system in two different scenarios to compare nonlearner agents with learner agents in both scenarios.