Expert Incorporated Deep Reinforcement Learning Approach for Market Arbitrage Strategy of the Battery Energy Storage

Peak-valley arbitrage is one of the important ways for energy storage systems to make profits. Traditional optimization methods have shortcomings such as long solution time, poor universality, and difficulty in applying to non-convex problems. This study addresses this issue by utilizing Deep Reinforcement Learning (DRL) to optimize the market arbitrage of battery storage system (BSS). Firstly, the market arbitrage problem is presented as a typical Markov Decision Process (MDP). Secondly, an expert incorporated DRL approach is proposed to seek for the optimal control strategy for the energy storage systems. Finally, the proposed algorithm is verified based on the U.K. electricity market background. The numerical simulation result show that the proposed method can achieve refined power control on the premise of preventing overcharge and over-discharge of the battery storage system, and achieve higher training efficiency comparing with existing deep reinforcement learning approach.