Deep Reinforcement Learning With a Look-Ahead Search for Robotic Cell Scheduling

Robotized manufacturing systems consisting of several processing machines and a robot for transporting jobs between the machines have been widely used in mechanical and electronic manufacturing industries. The sequence of robot tasks in such a robotized manufacturing system affects its productivity significantly, which also has the large impact on the overall production line consisting of multiple robotized manufacturing systems. This article addresses the scheduling problem in a single-gripper robotic cell, one of a robotized manufacturing systems. The objective is to minimize the makespan. To achieve this, a novel reinforcement learning method is proposed, which combines a look-ahead search (LAS) to improve decisionmaking using more accurate estimated makespan. Experimental results demonstrate the superior performance of the proposed method compared to existing approaches. Moreover, the method is applicable in dynamic environments with uncertain processing times.