A Switching Strategy for Run-to-Run Control Using Deep Deterministic Policy Gradient Algorithm and Integral Controller

An end-to-end strategy based on the deep determin-istic policy gradient (DDPG) algorithm is an effective adaptive method for run-to-run control, which allows exploration in a semiconductor manufacturing environment to overcome the limitations of the model. However, the tracking performance of DDPG can be compromised by inefficient sampling and inadequate training. To address such issues, a switching control strategy based on DDPG and integral controller is proposed. First, the target tracking is built as a Markov decision process. Given the current state, two actions are obtained from the integral controller and the DDPG, respectively. Then, an optimal action that provides a greater reward is selected as the output of the switching strategy. Therefore, the developed scheme not only guarantees the basic performance of the integral controller, but also fully exploits the exploration advantages of DDPG in unknown manufacturing environments. Simulation results demonstrate the improvement in tracking performance of the presented strategy compared to the standalone DDPG and integral controllers.