Action Masked Deep Reinforcement learning for Controlling Industrial Assembly Lines

The scalability of control algorithms controlling industrial assembly lines either model-based or model-free is still a big barrier from the point of view of implementation. In this paper, various Deep Reinforcement learning (DRL) algorithms will be tested in a virtual environment of a generic industrial assembly line in terms of optimally with respect to optimal exact solutions, and computational time. The agents will take bigger rewards when they minimize the finishing time while maintaining constraints, such as time precedence between tasks. The control actions will be modeled as a task assignment matrix that assigns the tasks to be completed to specific workstations. Another control action is the resource allocation. Due to the big number of constraints, many actions will be unfeasible leading to a longer time of training of the agent to learn not to apply unfeasible actions, therefore action masking will be introduced to the DRL algorithms to reduce the action space only to the feasible actions, thereby significantly reducing training time. Proximal Policy Optimization (PPO) agent shows faster and more stable convergence to the optimal solution over the Deep-Q Network (DQN) family, including the double DQN and Dueling one.