Deep Reinforcement Learning With Integer Optimization for Dynamic Slab Assignment Problem

This article investigates a dynamic slab assignment problem (DSAP) that arises in the slab production process of steel industry. In DSAP, a set of slabs and orders arrive dynamically at each time step of a planning period, and their information cannot be observed in advance. For a planning period, a series of decisions need to be made on allocating the slabs to customer orders, self-designed orders, or holding them in inventory to maximize total rewards. To address DSAP effectively, we formulate a Markov decision process (MDP) model and propose a deep reinforcement learning algorithm combined with an integer programming (DRLIP) model. DRLIP decomposes each decision time step into two stages, i.e., dynamic selection stage and static assignment stage. The dynamic selection stage primarily uses a double-pointer network (DPN) to select the slabs and orders to be involved in matching. In the static assignment stage, an extension of a multiknapsack problem is constructed based on the selected slabs and orders. We formulate an integer programming (IP) model to solve this multiknapsack problem for obtaining an optimal assignment decision, which in turn provides a reward for each time step. To evaluate the effectiveness of DRLIP, we use a global method, three advanced heuristic methods, and a scenario tree method for comparison on practical and randomly generated problem instances. Computational results show that DRLIP yields a mean gap of 21.5% versus the optimum from the global method and outperforms the other comparison methods.