Optimization of the Ice Storage Air Conditioning System Operation Based on Deep Reinforcement Learning

With the intention of obtaining the room temperature and economic cost control strategy of an ice storage air conditioning system in a small office building in Shanghai, the ice storage air conditioning system is established in this paper as a Markov decision process model and deep reinforcement learning algorithms are adopted to optimize its operation. In order to avoid the problem of over-dimension and over-estimation of value function caused by reinforcement learning, the DDQN (Double Deep Q-Network) algorithm with dual neural network structure is adopted to optimize the operation of ice storage air-conditioning system. Aiming at overcoming the shortcoming of slow convergence of DDQN algorithm, the action space of DDQN is taken into consideration in this paper. Firstly, the appropriate action set is selected according to the convergence speed of different actions. Secondly, the exponential function is addressed in the reward function. Based on the exponential function, the reward function can adjust the penalty value according to the difference between the expected room temperature and the actual room temperature, thus speeding up the convergence of the DDQN algorithm. Finally, Python is adopted to model and simulate buildings and ice storage air conditioning systems. The simulation results show that the operating cost and the proportion of uncomfortable time are both reduced by DDQN control method proposed in this paper with better control performance.