Optimizing Control of Wastewater Treatment Plant With Reinforcement Learning: Technical Evaluation of Twin-Delayed Deep Deterministic Policy Gradient Agent

Control of the wastewater treatment processes presents significant challenges due to the fluctuating nature of inflow and wastewater composition, alongside the system’s non-linear dynamics. Traditional control methods struggle to adapt to these variations, leading to an economically suboptimal operation of the process and a violation of norms imposed on the quality of wastewater discharged to the catchment area. This study proposes using a reinforcement learning approach to control a wastewater treatment plant. Specifically, the twin-delayed deep deterministic policy gradient algorithm is employed and comprehensively validated. This algorithm enhances control efficiency, optimizing costs while improving the plant’s wastewater treatment capabilities. The proposed control approach is implemented and evaluated using Benchmark Simulation Model No. 1. The evaluation is based on four representative weather scenarios and examines sixteen different metrics. In the proposed case study, an agent controls oxygen transfer coefficients in three oxygen tanks of a wastewater treatment plant. Two distinct reward functions are employed to guide the reinforcement learning agent. The first one focuses on maintaining aerobic conditions and minimizing total nitrogen. The second function retains these objectives and also aims to minimize the aeration energy. The experimental results demonstrate that the proposed control strategy significantly enhances aeration energy efficiency and reduces the overall cost index, indicating a more cost-effective operation. However, future work is required to improve further the performance of reinforcement learning that optimizes the wastewater treatment process.