Obtaining an Object’s 3D Model Using Dual-Arm Robotic Manipulation and Stationary Depth Sensing

When humans want to understand an object’s 3D shape, they watch the object from different viewpoints. Changing the viewpoint is either performed actively, i.e., moving eye sights or the human head, or passively, i.e., holding and reposing the object. Inspired by the humans’ passive policy, we propose a method to plan the motion for a dual-arm robot to hold and repose an object, capture multiple views using a stationary depth sensor mounted on the robot head, and obtain the object’s 3D shape from the multiple views. Primarily, we develop algorithms to determine the Next Best Configuration (NBC) for observation and Next Best Regrasp/Grasp (NBR/G) poses while considering elements like the confidence of captured partial point clouds, robotic manipulability, robotic motion distances, and sensing ranges. We study the necessity and influence of these elements on the time costs and surface coverage quality in the experimental section using several representative objects. The results show that the elements play essential roles in supporting specific actions or suppressing certain costs. They help to secure efficient robot motion and satisfactory 3D shape recovery quality.
Note to Practitioners
—This paper is motivated by the difficulties in using commercial 3D scanners. A commercial 3D scanner set usually comprises a scanning sensor, a rotating table, and editing software. To scan the 3D shape of an object, a human needs to place the object on the rotating table with different poses, let the scanner obtain several partial point clouds, and use the editing software to merge them into a final model. The human must carefully design the different poses by considering both the object’s self obstructions and stable placements, which is tiring and difficult to be applied to large-scale tasks like building 3D shape databases containing many objects. On the other hand, although several robotic solutions exist for automatic scanning, they either use an eye-in-hand scanner to scan a stationary object or an arm to hold and move an object for scanning. In the former case, the bottom or downward faces of the object cannot be covered. In the latter case, the surface blocked by the fingers of the holding hand will be lost. The method proposed by this paper plans dual-arm robot motion to grasp and move objects for scanning. It automatically determines pick-up, rotation, and handover to maximize scanning coverage. Compared with commercial scanners and existing robotic solutions, the method performs automatic scanning with high coverage and is more advantageous for scanning many objects without human intervention.