Dual Stocker Scheduling Optimization Based on Reinforcement Learning

The dual stocker scheduling (DSS) problem aims to optimize the ‘stocker’ used in the automatic material handling system (AMHS), which is a system mainly used in semiconductors (wafers) or display manufacturing processes. Dual crane stockers can save transportation time depending on the order in which products are transported, as they gather information about the location and destination of the product to be transported. Therefore, the purpose of DSS is to minimize this transport time, which is called makespan. To solve this problem, we developed a dynamic programming in deep Q-network (DPDQN) algorithm that can minimize the time taken for cranes to move within the factory and complete tasks in a shorter time by partially improving the shortcomings of the existing dynamic programming + deep Q-network (DP+DQN) algorithm. Additionally, zero weights are assigned to the inputs of the network to allow flexible use of the number of jobs that can be changed depending on the situation. Furthermore, in consideration of handoffs that could be caused by the dedicated spaces of the cranes in the actual process environments, the algorithm is designed to be applicable to real industrial scenarios.