Seasonal Performance of Fitted Q-iteration for Space Heating and Cooling with Heat Pumps

The majority of building energy consumption is related to space heating; advanced control algorithms allow to reduce the associated costs. This work uses Fitted Q-iteration, (FQI) a model-free reinforcement learning method, to control a heat pump for space heating and cooling throughout the year. The naive FQI implementation is modified to exploit seasonality in heat pump load patterns by restricting the action space based on the season. The operating costs are compared against a rule-based controller (RBC) and a model-predictive controller. It was found that FQI performs well during winter and summer, but performs notably worse than RBC during more temperate days in spring and autumn. In the initial training period at the start of each season, costs are significantly higher than for RBC. These costs can be reduced by pre-training on data samples from past season.