Energy Optimization for Building Energy Management with Thermal Storage: A Multi-Agent Deep Reinforcement Learning Approach

Building Energy Management (BEM) with Thermal Energy Storage (TES) poses significant challenges due to the intricate coordination required among components such as Power-to-Heat (P2H) converters, TES units, and zone temperature controllers. In this paper, we propose a novel multi-agent Deep Reinforcement Learning (DRL) method for BEM, capable of optimizing the energy efficiency and flexibility of Building Energy Systems (BESs) integrated with TES without requiring manual efforts to set up a control-oriented model. Our method-distributed optimized NNPC via DRL-consists of two parts: i) a Neural Network Predictive Control (NNPC) for zone temperature regulation, leveraging an attention-based artificial neural network for forecasting operational conditions and a Dynamic Programming (DP) algorithm to jointly optimize both comfort levels and energy efficiency; and ii) a distributed optimization strategy realized through a coordinating policy that facilitates the collaboration between conventional on/off and NNPC agents/controllers. The efficacy of our method is confirmed through simulations established based on a real-world BES that integrates a TES unit. We compare our method with three alternatives: a conventional on/off control, an optimized on/off control, and an optimized Naïve NNPC, with the latter two also incorporating our distributed optimization strategy. Specifically, our method realizes approximately 7% reduction in energy consumption compared to the conventional on-off method while maintaining equivalent comfort levels. Furthermore, it attains parity with the performance of the optimized Naïve NNPC without requiring manual tuning, thereby illustrating its adaptability and ease of implementation. Collectively, these findings underscore our method’s superiority across metrics of energy efficiency, comfort and applicability, demonstrating the profound potential of DRL in BEM, particularly when TES is involved.