Strategic Bidding and Trading in Joint Electricity-Carbon Market with Deep Reinforcement Learning

Generation companies (GenCos) participate in both the electricity and carbon markets. Against this backdrop, coordinated strategic actions provide flexibility when maximizing GenCo's profit in joint electricity-carbon markets. However, dynamic coupling relationship in joint markets and uncertainties of renewable energy sources poses challenges. In this paper, a reinforcement learning (RL)-based multi-market joint coordination method is proposed. We first formulate the joint electricity-carbon market coordination problem and then design a partially observable Markov decision process (POMDP) on the problem. Then, a multi-market proximal policy optimization (MM-PPO) is adopted to solve the problem. Experiments show that by coordinating their bidding and trading strategies, GenCo can achieve higher profits than in the case where they act independently and also demonstrate its effectiveness in handling uncertainties in the joint electricity-carbon markets.