QMIX-Based Multi-Agent Reinforcement Learning for Electric Vehicle-Facilitated Peak Shaving

Given the energy storage capacity and rapid response capabilities, electric vehicles (EVs) hold the potential to offer auxiliary services, including peak shaving and frequency regulation, to the power grid during emergencies. Nevertheless, effectively coordinating EVs presents a complex challenge for multiple charging stations (CSs), which must integrate dynamic traffic fluctuations into a schedule while ensuring sufficient power resources for these auxiliary services. To tackle these challenges, this paper proposes an across-realm decision-making framework to dispatch EVs to CSs for peak shaving considering dynamic traffic conditions and power constraints. We develop a cooperative multi-agent reinforcement learning (MARL) strategy, which capitalizes on the collaboration among CSs by formulating the EV dispatching problem as a Markov Game. The CSs are regarded as learning agents and a QMIX network with bidding mechanism is applied to train the joint action towards maximizing long-term team rewards. The proposed method has been tested and compared to the existing reinforcement learning and non-reinforcement learning methods, and the simulation results have demonstrated the efficiency of the proposed approach considering real-world scenarios.