Multi-Objective Scheduling for Green Flexible Assembly Job-Shop System via Multi-Agent Deep Reinforcement Learning With Game Theory

In discrete manufacturing systems, production scheduling techniques are urgently needed with lacking dynamic response-ability, inferior real-time decision-making, and sharply increasing uncertainty in the production process. In this regard, this paper presents the flexible assembly job-shop scheduling (FAJS) problem that incorporates job shop and assembly shop. And, a multi-agent deep reinforcement learning (MA-DRL) system with game theory is proposed to solve the proposed FAJS problem aiming at minimizing the Makespan, total energy consumption, and human factor’s comfort. A mathematical model is formulated to describe the FAJS problem, which then is translated into a Markov Decision Process (MDP) where an agent directly selects behavioral policies according to the processing state of the current decision point. The processing state feature data that uses a deep convolutional neural network to fit the value function is extracted from three matrices including the processing time matrix, task assignment Boolean matrix, and an adjacency matrix. Simple Constructive Heuristics (SCH) are used as candidate action for scheduling decisions. For multi-objective optimization into a reward strategy problem, a game model combining Nash equilibrium and Pareto optimality is established to obtain an objective unification solution. A multi-agent deep deterministic policy gradient (MA-DDPG) framework is designed to train the proposed MA-DRL model. The case ‘MK’ with four assembly constraints is formed as the novel datasets. These datasets and a case of the plunger pump production workshop scheduling scenario are designed to analyze the basic performance of this algorithm. The results show that the proposed algorithm compares with other single-indicator optimal algorithms with 9.47 per cent, 12.66 per cent and 22.86 per cent respectively for the three-evaluation metrics (GD, and HV), which contributed to solving the practical production problem of the manufacturing industry.