Energy Management of Microgrid with Electric-Hydrogen Hybrid Energy Storage System Based on an Improved SAC Algorithm

Hydrogen energy, as a low-carbon renewable energy source and a new raw material, plays a crucial role in the energy transition and serves as an important complement to electric power in the secondary energy system. Deep reinforcement learning (DRL) has proven to be advantageous in solving optimization problems for microgrids, which are often subject to substantial uncertainty. This paper proposes a microgrid model with an electric-hydrogen hybrid energy storage system (EH-HESS), aimed at achieving energy management for the microgrid and addressing its seasonal fluctuations. Firstly, a mathematical model of the microgrid with EH-HESS is developed, and the scheduling process is formalized as a Markov decision process (MDP). Secondly, an improved soft actor-critic (SAC) algorithm is proposed for training, which innovatively incorporates prioritized experience replay (PER) with the exploration-exploitation trade-off (EET) into SAC. Throughout the training process, the algorithm adaptively adjusts the weights of temporal difference (TD) and Bellman (BE) errors in the sampling probabilities, thereby achieving EET and improving training efficiency. Finally, the scheduling results for typical days across different seasons validate the feasibility and effectiveness of the proposed method.