Controlling Pressure of Gas Pipeline Network Based on Mixed Proximal Policy Optimization

The gas pipeline network plays a significant role in transportation due to its low cost and safe operation. However, the traditional gas pipeline network control scheme has been criticized for its inefficiency. Reinforcement learning (RL) provides an emerging model-free alternative, but its poor generalization performance becomes a practical obstacle in diverse equipment scenarios. In this paper, a Mixed Proximal Policy Optimization (PPO) approach, Mixed-PPO, is proposed to simultaneously control the continuous and discrete equipment. Compared with the original PPO, the proposed method exhibits better performance in the following aspects: (i) the ability to realize the control of both continuous and discrete equipment, making gas pipeline network pressure more stable; (ii) quicker convergence while obtaining higher profits. The effectiveness of our method is illustrated by a case study. The results show that Mixed-PPO outperforms the original PPO in terms of both profit increase and pressure fluctuation decrease, by 76.86% and 38.65%, respectively.