Automatic Generation of Rescheduling Knowledge in Socio-technical Manufacturing Systems using Deep Reinforcement Learning

The generation of rescheduling knowledge for handling unforeseen events has become a key element of any real-time disruption management strategy, to ensuring a highly efficient production in increasing dynamic conditions without sacrificing cost effectiveness, product quality and on-time delivery, which are key competences in modern socio-technical manufacturing systems characterised by a diminishing predictability of environmental conditions at the shop-floor. In this work, a real-time rescheduling task is modelled and solved resorting to the integration of a schedule state simulator with an artificial agent that can learn successful schedule repairing policies directly from high-dimensional sensory inputs. The rescheduling knowledge is stored in a deep Q-network, which can be used reactively to select repair actions in order to make progress toward a goal schedule state. The network is trained using deep Q-learning with experience replay over a variety of simulated transitions between schedule states using the schedule visual images and negligible prior knowledge as input. Finally, an industrial example is discussed showing that the approach enables learning successful rescheduling policies and encoding task-specific knowledge that can be understood by human experts.