Reinforcement Learning-Based Strategic Bidding for Generation Companies in Electricity Markets

In the context of strategic bidding in competitive electricity markets, deregulated electricity markets refer to a market structure where multiple power providers compete strategically to sell electricity to retail customers. Deep Deterministic Policy Gradient (DDPG) has been extensively investigated as a reinforcement learning technique for continuous control problems, including power market bidding tactics. It has shown promising results in optimizing the decision-making processes of Generation Companies (GenCos) in these markets. This paper presents TD3 (Twin Delayed Deep Deterministic Policy Gradient), a new approach in deep reinforcement learning (DRL) that improves upon DDPG. TD3 incorporates twin critics, smoothed target policy, and delayed policy updates. The proposed TD3 algorithm aims to maximize GenCo profit and foster a healthy competitive environment, addressing the challenges faced by GenCos in bidding methods for energy markets. By bridging a research gap, this algorithm demonstrates significant improvements over previous strategies by leveraging the capabilities of TD3, enabling corporations to navigate the complex dynamics of power markets more successfully. Illustrative scenarios conducted on the IEEE 30-Bus test system using PyTorch showcase the superior profitability of the TD3 method for GenCos in power markets compared to older RL methods. The results highlight the potential of TD3 as a viable solution to enhance GenCo bidding methods and provide a more effective means of navigating the intricacies of power markets.