Meta Reinforcement Learning Based Adaptive and Interpretable Energy Storage Control Meets Dynamic Scenarios

As renewable energy becomes more widespread, energy storage systems (ESSs) play an important role in managing energy distribution and economic arbitrage. Traditional reinforcement learning (RL)-based scheduling methods face performance degradation or failure in highly dynamic environments due to their limited generalization capability, especially amid increasing fluctuations in renewable energy output, electricity prices, and load demands. This paper proposes a novel ESSs control framework based on Meta-Reinforcement Learning (Meta-RL), comprising offline training and online adaptation phases. The offline training phase features a dual-loop update framework, acquiring a model with highly generalizable initial parameters through a multi-task learning approach. In the online adaptation phase, the model can make adaptive adjustments rapidly in response to environmental changes. Lastly, a Shapley Value-based Meta-RL (SMRL) method is proposed to perform an in-depth analysis of decision-making outcomes. Multiple dynamic microgrid (MG) scenarios were simulated using real data with added fluctuation factors for model training and testing. The proposed Meta-RL based model exhibits significantly better adaptability in various scenarios, achieving approximately 20% to 50% performance improvement compared to the traditional RL model. A detailed analysis of the contribution of features behind scheduling decisions shows alignment with human intuition.