Multi-Time-Scale Joint Optimization of Data Center Market Transactions and Computing Resource Allocation

Due to the uncertainty of renewable energy generation, frequency fluctuations and supply-demand imbalance issues of the power grid are becoming increasingly prominent. Since computing jobs can be allocated varying amounts of resources within service level agreement constraints, the data centers possess temporal flexibility in energy consumption. By leveraging this flexibility, data centers can support the grid by participating in ancillary service markets while earning additional revenue. However, the multi-time-scale requirements of grid regulation demand create challenges for joint market bidding and resource allocation strategies. In this paper, we propose a multi-time-scale decision-making approach for data center bidding and resource allocation to provide both frequency regulation and reserve services. First, we model the hour-ahead market bid, real-time power decision, and real-time resource allocation processes as three-layer Markov Decision Processes. Next, we develop a multi-time-scale optimization method based on hierarchical reinforcement learning to solve the proposed model. At the upper layer, the market bid decision is optimized using the Deep Deterministic Policy Gradient algorithm. Based on the bidding, the second layer determines the target operation power. Based on the target power, the third layer optimizes job resource allocation. Finally, we validate the effectiveness of the proposed method for data center market bidding and resource allocation through numerical experiments and analysis.