Towards Retail Stores Automation: 6-DOF Pose Estimation Combining Deep Learning Object Detection and Dense Depth Alignment

Automating in-store logistics processes in the retail industry poses significant challenges for robot manipulators. Contrary to warehouses, retails stores are subject to customer actions, which can imply non-standard tidying of products. This paper addresses the problem of detecting, discriminating, and accurately estimating the 6 degrees-of-freedom (6-DOF) pose of individual products, even in unexpected positions such as fallen or wrongly placed objects. The trained object detection model successfully discriminated similar-shaped objects of different brands/types commonly found in convenience stores. The detection is used to initialized the object position while several possible orientations are explored by a Fibonacci Multi-Start method. The estimated pose is then refined by a multi-scale projective Iterative Closest Point (ICP). The evaluation of the complete 6-DOF pose estimation module revealed its consistent ability to converge to the correct pose, avoiding local optima and achieving sub-millimetric precision. A working demonstration is presented, showcasing a robot rearranging a convenience store shelf. The overall system demonstrated the ability to detect fallen objects, estimate their poses, determine suitable grasping directions, and execute successful grasps. Importantly, the system's feasibility with minimal human intervention was demonstrated, allowing easy addition of new objects by convenience store employees or other stakeholders.