Action Decomposition-based Actor-Critic for Supply Chain Optimization

In recent years, deep reinforcement learning (DRL) has demonstrated significant potential to address complex and dynamic supply chain optimization (SCO) problems. However, existing DRL algorithms often encounter challenges when dealing with large-scale and high-dimensional supply chain decisions, making it difficult to effectively coordinate production and transportation. To address these issues, this paper proposes an innovative deep reinforcement learning algorithm—Action Decomposition Actor-Critic (ADAC). This algorithm significantly reduces learning complexity by decomposing complex decision tasks into multiple subtasks. Based on real-world supply chain scenarios, we construct a complex multi-stage supply chain environment. Additionally, we employ action decomposition-based Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) algorithms to learn optimal policies in continuous action spaces, enabling fine-grained control of production and transportation. To verify the effectiveness of the algorithm, we conduct extensive experiments on both simulation and real-world datasets. Experimental results demonstrate that the ADAC algorithm outperforms traditional heuristic algorithms and general DRL algorithms in multiple complex supply chain scenarios. This shows the strong robustness and wide applicability of the ADAC algorithm in SCO problems.