Enhancing Robotics Online 3D Bin Packing: A Comparative Study of Conventional Heuristic and Deep Reinforcement Learning Approaches

Robotic object packing holds a wide array of practical applications across logistics and manufacturing sectors. The online 3D Bin Packing Problem (BPP) is a challenging task that involves online packing of three-dimensional boxes into a container while considering constraints and objectives. Unlike the offline version, where all boxes are known in advance, the online variant requires decisions about how to pack each box as it arrives, without prior knowledge of upcoming boxes.To maximize space utilization, our study explores two distinct strategies: conventional heuristics-based algorithms and a deep reinforcement learning (DRL)-based approach. For the heuristic strategy, we propose four heuristic criteria alongside two variants of multi-objective optimization (MOO) algorithms. Quantitative experiments reveal that MOO outperforms single-objective approaches for online bin packing. In our DRL-based approach, we introduce a framework that leverages a candidate map that indicates the potentially feasible placements, ensuring a balanced exploration and exploitation in the considerable discrete action space. Experiments demonstrate the superior performance of our DRL-based approach compared to both DRL-based baseline methods and conventional approaches. Additionally, we discuss the limitations of DRL-based methods and offer practical recommendations for real-world applications.