Reinforcement learning based mass flow and supply temperature control for combined heat distribution

Combined heat distribution circuits (CHDCs) are increasingly used in apartment buildings. Here only one supply pipe distributes both space heating (SH) and domestic hot water (DHW). Currently, the supply temperature is set to the highest temperature needed by one of the end-users (i.e. 65ºC for DHW), even if low-temperature emitters are used for SH. However, using decentral storage tanks for DHW enable demand-based temperature controls to reduce unnecessary heat losses and poor efficiencies. This research uses reinforcement learning (RL), a machine learning technique, to develop new control strategies for CHDCs with underfloor heating and DHW storage tanks. The agent controls the supply temperature and the mass flow in the hybrid boiler room. Whether the RL agent is able to find the optimal control strategy depends on the definition of its Markov Decision Process (MDP) model elements, namely the states, the possible control actions and the reward function. The results show that an increasing gamma and decreasing learning rate during training leads to better performance and that the agent with the largest flexibility develops a better control strategy that resulted in up to 23% primary energy savings.