Job Shop Scheduling Problem Using Proximal Policy Optimization

Job shop scheduling in dynamic environments has been a challenging problem in manufacturing system. Meanwhile, in the backdrop of Industry 4.0, developing smart scheduling systems capable of real-time decision-making and rapid responsiveness holds significant relevance. Hence, this paper delves into the realm of dynamic job shop scheduling, where job arrivals occur randomly. It introduces a real-time smart scheduling framework grounded in deep reinforcement learning. The scheduling agent is trained through proximal policy optimization, while the environment is built upon discrete event simulation. Additionally, a structured approach is devised, encompassing independent states, actions, and rewards for decision-making. To enhance convergence performance, reward scaling techniques are implemented. Extensive numerical experiments are conducted across various production configurations. The results demonstrate that the PPO-based approach can outperform dispatch rules and two hyper-heuristics in most untrained scenarios within a comparable computational time frame. This validates the effectiveness, scalability, and generalization capability of the proposed method.