Asynchronous Advantage Actor-Critic Based Approach for Economic Optimization in the Integrated Energy System with Energy Hub

With the further integration of advanced information technology and energy internet, it is necessary to apply data-driven methods to solve complicated optimization problems. In this paper, an integrated power, natural gas and heating system is selected as the research object, and energy hub is used to provide the flexibility of energy interaction. This work aims to decide the optimal energy allocation ratio of the energy hub to minimize the operation costs. However, considering diversify uncertainties, such as intermittent of wind power, randomness of load demand and energy price, the optimal energy allocation problem is formulated as a Markov decision process. Then, the powerful deep reinforcement learning algorithm, asynchronous advantage actor-critic, is applied to make the real-time decision. The optimal energy allocation ratio can be achieved by plenty of training iterations. Simulation is conducted on the combination of IEEE 33-bus power system, 17-node heating system and 14-node gas system. The results demonstrate that the effectiveness and superiority of the asynchronous advantage actor-critic -based energy allocation strategy.