A DQN-Based Coordination Method of HVAC and Energy Storage for Building Energy Management

Buildings account for 33% of global energy consumption and generate 28%of carbon emissions, drawing increasing attention in energy sector. However, the complex building thermal dynamic process poses a great challenge for building energy management (BEM). Thus, this paper proposes a coordinated BEM method of HVAC and energy storage based on deep Q-network (DQN), aiming to reduce the building operating cost while maintaining occupant comfort. Also, a variable-length rolling-horizon procedure is applied to enhance the coupling of the future scheduling time interval. In this method, a third-order building equivalent thermal model is integrated into the DQN to accurately model the dynamic temperature changes in the building. Moreover, considering the uncertainties of photovoltaic (PV) outputs and loads, the concept of stochastic optimization is introduced to the state and reward functions of DQN by generating multiple scenarios based on prediction data. The Simulation results verify that the proposed DQN-based BEM method can effectively reduce the building operating cost and keep a high level of comfort throughout the day by coordinating HVAC and energy storage.