A New Deep Reinforcement Learning Run-to-Run Control Algorithm for Mixed-Product Production Mode in Semiconductor Manufacturing

This paper proposes a new Run-to-Run (R2R) control framework based on deep deterministic policy gradient (DDPG) for the mixed-product production mode in semiconductor manufacturing. The DDPG algorithm is particularly developed to configure a deep reinforcement learning environment well suited to mixed-product production modes. To address the challenges posed in deep reinforcement learning, three enhanced mechanisms have been developed to improve the training of the proposed DDPG model for mixed-product R2R applications. These mechanisms include a piece-wise reward function, training with dynamic targets, and the new recall principle. It is demonstrated from the comprehensive simulation results that the proposed R2R control framework outperforms five noted mixed-product R2R control algorithms in the literature. The research outcome of this paper signifies a promising viability of deep reinforcement learning for highly complex and dynamic environments with continuous action spaces in the mixed-product R2R practice.