A Hybrid Method Based on Multi-Agent Reinforcement Learning and Integer Programming for Dynamic Slab Design Problems in Steel Industry

This paper investigates a dynamic slab design problem in the steel industry, where order demands arrive dynamically during a given period. Slabs are the raw materials for producing order plates demanded by customers—slabs are first rolled in a rolling mill to create desired mother plates, and then the mother plates are cut into order plates. The dynamic nature of orders, along with practical considerations regarding rolling methods and nonlinear size constraints, distinguish our problem from existing ones. The goal is to determine slab design schemes to fulfill order demands for the period. However, the stochastic nature of dynamic production and the inherent complexity of slab design present significant challenges in the efficient solution. To address these challenges, we formulate a Partially Observable Markov Decision Process (POMDP) and propose a hybrid method (MARLIP) based on multi-agent reinforcement learning (MARL) and integer programming. The MARL component determines the order plate set, the slab type, and the rolling method involved in a single slab design. Then, an integer programming model is constructed to optimize the arrangement of the order plates based on MARL’s decisions. To further refine the MARLIP approach, we introduce dual factors and a soft boundary into the MARL and propose a two-core-based dynamic programming method to solve the integer programming model. Experimental results demonstrate our algorithm’s superior performance compared to several competitive reinforcement learning and mathematical programming methods in the dynamic slab design problem. Note to Practitioners—This paper is motivated by the medium-heavy plate production in the steel industry, where slabs are rolled into mother plates, and order plates required by customers are cut from the mother plates. This study is also applicable to other industries for which larger objects need cutting into smaller ones, such as the glass industry and paper industry. Current research mainly focuses on slab design for known batches of orders but falls short when dealing with dynamically arriving orders. We formulate a Partial Markov Decision Process to link slab design decisions over time, aiming to maximize the total slab utilization of slab designs for the given period. Then, we propose a hybrid method named MARLIP to maximize the total slab utilization of a given period by dynamically determining slab design schemes. At each time step, MARLIP determines a slab for producing order plates, a rolling method for rolling the slab into a mother plate, and the optimal arrangement of order plates on the mother plate. Experimental results indicate that MARLIP increases total profit, reduces total cutting loss, and utilizes fewer slabs to fulfill orders. By making rapid and automated decisions for dynamically arriving orders, MARLIP enables smart and automated production in the steel industry. In the future, we will possibly extend the proposed method to dynamic cutting and packing problems in manufacturing and then promote more practical applications.