Deep Reinforcement Learning-Based Dynamic Reconfiguration Planning for Digital Twin-Driven Smart Manufacturing Systems With Reconfigurable Machine Tools

Smart manufacturing systems are a new paradigm in Industry 4.0 driven by the emerging information and communication technology and artificial intelligence that converge to digital twin, which are able to perceive, recognize, and handle the changes in demand and production. Reconfigurable machine tools (RMTs) can promote the flexibility of smart manufacturing systems. The fundamental problem lies in dynamically reconfiguring the RMTs in smart manufacturing systems efficiently and accurately by considering the flexibility of production precedence and operation sequences simultaneously. Therefore, in this article, a deep reinforcement learning-based reconfiguration planning method of digital twin-driven smart manufacturing systems with RMT is proposed to seek optimal reconfiguration policy online. The reconfiguration processes of smart manufacturing systems are modeled by considering reconfiguration cost, moving cost, and processing cost. Deep
Q
-network is adopted to explore the state space and action space to find the optimal reconfiguration scheme with the highest return. An industry case study is presented to demonstrate the effectiveness and efficiency of the proposed method, where the reconfiguration processes of a smart manufacturing system consisting of five RMTs for producing four parts are discussed.