Comfort and energy management of multi-zone HVAC system based on Multi-Agent Deep Reinforcement Learning

Energy consumption generated by HVAC systems accounts for more than 40% of total energy usage in commercial buildings. There are many studies focusing on improving comfort conditions for occupants in multi-zones while reducing energy consumption. For example, the widely used Multi-agent Deep Deterministic Policy Gradients algorithm, however, the principle of centralized training and overestimation strategy, which will make the algorithm fall into a sub-optimal process and have the slowly training speed. In order to solve this problem, we decided to adopt multi-agent twin-delayed deep deterministic policy gradient algorithm for the HVAC control of multi zone building. In this work, we assume that one zone is controlled by its corresponding agent and consider the control problem as a Markov game first. Then we adopt delayed update strategy and design a target policy smoothing technique for our proposed method. Furthermore, two value functions will be designed for each agent to prevent overestimation. We validate the effectiveness of the method for HVAC systems by simulation using TRNSYS software.