Reinforcement Learning with Hierarchical Graph Structure for Flexible Job Shop Scheduling

The flexible job shop scheduling problem (FJSP) is a NP-hard combinatorial optimization challenge, having extensive real-world industrial applications. For a given instance of FJSP, a scheduling solution need to be devised, which entails determining the appropriate machine for each operation and establishing the sequence in which operations are processed on a given machine, all while adhering to certain constraints. However, many of the methods are lacking in flexibility, unable to cope effectively with multi-relational operations and machine selection scenarios. This can make the representation of state and action challenging in practice, leading to a significant increase in decision complexity. This paper considers these issues by a hierarchical graph structure with a model that synergistically integrates two modules: the environment and the decision module. The environment module is characterized by a hierarchical graph neural network which defines the multi-relational FJSP instance where all machine and operation data are stored. The decision module incorporates the Actor-Critic networks, designed to train reasoning strategies and enhance decision performance, based on the states and rewards provided at each step by the environment module. The objective is to derive optimal scheduling strategies through interactions between the decision agent and the environment, targeting the minimum resources consumed or makespan. Numerical simulations validate the efficacy of the proposed approach, showing its potential to yield superior results and thereby enhance performance.