Interval Weight State Update Approach with Deep Reinforcement Learning for Dynamic Load Change Tasks in Air Separation Processes

The air separation unit (ASU) is important for producing high-purity oxygen, nitrogen, and argon, and dynamic load change operation helps meet the needs of downstream industries. The ASU has an extremely long delay, and the requirements for this operation are strict, and model predictive control (MPC) is usually used for this task. However, the burden of online MPC computation is a key issue in improving production and economic efficiency. In this paper, a framework based on Reinforcement Learning (RL) is proposed to solve the long-delayed task in ASU. The state and reward function are specially designed for this task. First, training increases the sampling time of the original model to create a low resolution environment, which makes training more efficient. As a sacrifice, the control precision is reduced in the original high resolution environment. Then, an algorithm called Interval Weighted State Update (IWSU) is proposed to restore the agent and keeps the action accuracy in the original environment. The dynamic load change task works as the case in this paper. The performance of RL is compared with that of MPC, and results show that RL is almost exactly the same as MPC in key flow indicators, and can well meet the remaining content and purity range requirements. IWSU shows good performance compared with directly applying the agent to the original high resolution environment.