Cross-Entropy Firefly Algorithm-Assisted Deep Reinforcement Learning for Flexible Flow Shop Scheduling Problems

The Flexible Flow Shop Scheduling Problem (FFSP) involves scheduling jobs across multiple machines with flexible operation sequences. As an NP-hard problem with discrete variables and complex constraints, FFSP presents significant challenges in production scheduling. To address this, a mathematical model is developed with the goal of minimizing completion time, and an MDP model is formulated for Deep Reinforcement Learning (DRL) to identify optimal scheduling policies. A Cross-Entropy Firefly Algorithm-assisted Deep Reinforcement Learning (CEFA-DRL) framework is proposed to handle the discrete nature and NP-hardness of FFSP. Compared to conventional DRL algorithms, CEFA-DRL reduces the agent’s reliance on gradient information, making it more effective for problems with highly discrete solution spaces. The algorithm combines the Cross-Entropy (CE) method and Firefly Algorithm (FA) for co-evolution, assisting DRL in exploring the strategy space. An innovative sampling strategy is introduced to balance exploration and exploitation during training. The CEFA-DRL framework optimizes the Actor-Critic architecture through evolutionary reinforcement learning, improving exploration in solution spaces with limited gradient information, which is particularly suitable for scheduling problems. Cross-environment comparative experiments show that CEFA-DRL reduces makespan by approximately 29%–70% and variance by 7%–17% compared to other DRL and metaheuristic methods.