Accelerating Reinforcement Learning for HVAC Systems Using an LSTM-based Surrogate Simulator

Reinforcement learning (RL) has shown great potential in optimizing the operation of HVAC systems, improving energy efficiency, and enhancing user comfort. However, the slow input/output operations associated with simulation tools like EnergyPlus significantly hinder the training process. This paper proposes a novel approach to accelerate RL training by using a data-driven LSTM model to replicate the behavior of a building energy simulator. By training the LSTM model on a set of observations and actions, the model learns to approximate the simulator’s dynamics, providing a faster and more efficient training environment for RL agents. We demonstrate that using the LSTM-based surrogate simulator leads to substantial reductions in computational time while maintaining the accuracy of the system’s behavior.