An Improved DDQN Algorithm for Microgrid Energy Management with Strict Constraints

Deep reinforcement learning (DRL) is one of the most promising data-based approaches in microgrid energy optimal scheduling. However, conventional DRL algorithms are incompetent to train effectively when facing strict constraints. Therefore, an improved double deep Q network (DDQN) al-gorithm is proposed. The reinforcement learning parameter ε declines according to a binary function related to both the episode index and the decision-making step ordinal, instead of following the unary linear function in conventional DDQN. Besides, the training process is divided into two stages named the pathfinding stage and the optimization stage in proper order. On the optimization stage, the action at each decision-making step is selected from the respective subset that has been chosen on the pathfinding stage. As a result, the training failure under strict constraints is avoided and the convergence of the DDQN algorithm is enhanced. The numerical simulation experiments verify the feasibility and effectiveness of the proposed method.