Interactive Integrated Design Framework for Optimizing the Structure, Capacity, and Operation of Multienergy Systems via Reinforcement Learning

The complex and varied source–load characteristics of multienergy systems (MESs) make it difficult to match supply with demand while maintaining economic efficiency. Previous design methods have not fully considered the coupled relationships between the system structure scheme and the equipment capacity and operation scheme, which has made it impossible for them to design the optimal MES in terms of overall performance. To address this issue, a bilevel interactive integrated design framework (IIDF) is proposed for MESs. A generalized description of the integrated design problem for MESs was formulated that fully considers the selection of devices and their connections. In the first level, and the structure search strategy is constructed based on reinforcement learning to determine the optimal system structure. To accelerate convergence, a search space is designed that narrows the solution domain of candidate devices and connection parameters by integrating domain knowledge of multienergy flow supply and demand balance and heterogeneous energy cascade transformation. In second level, the capacity–operation co–optimization is executed for the determined MES structure from the first level. The performance of IIDF was evaluated against two traditional design methods in three typical scenarios, and the results demonstrated its effectiveness and superiority.