Development and Deployment of Reinforcement Learning based Control for Large Scale Decision-Making in Manufacturing: A Case Study

We investigate Deep Reinforcement Learning (DRL) for a large scale manufacturing problem of pallet loop control in an automotive assembly plant with multiple body types assembled in the same facility. The pallet loop comprises of three body construction lines that merges into a complex shared conveyor system and delivers units to the paint shop with empty pallets returning back to the start of construction lines. The decision problem is to dynamically control pallet selection at the various merge points to meet, often highly variable, daily delivery targets while being robust to unanticipated production downtime, part shortages and failures. We discuss the challenges associated with this problem and present solutions developed to successfully train and deploy a DRL control policy. Specifically, we introduce concepts of transient and steady-state reward functions required to address the problem of delayed rewards owing to spatially distributed and temporally shifted control locations. Simulation and real world deployment results show that the trained DRL policy outperforms human designed decision rules in that $\mathbf{i}$) it discovered a control policy with several benefits over base human designed decisions, ii) it demonstrated robustness against uncertain downtime in body construction lines, and iii) it was resilient to variations in delivery targets. We present results from plant performance before and after the implementation of the proposed DRL control policy.