Monocular visual pose estimation via online sampling for Mars sample-tube pickup

The 2011 Planetary Science Decadal Survey identified making significant progress towards Mars sample return (MSR) as a top priority for NASA, where a three-mission concept for MSR is currently being investigated. The Mars 2020 mission is intended to collect samples which will be sealed in tubes and left on the surface for potential return to Earth by a future mission. This paper studies the problem, autonomous pick-up of sample tubes using a monocular camera attached to a robot's end-effector. We estimate the pose of the tube and the gripper in a single image where both are visible, and compute incremental arm motions based on the relative transformation between the tube and the gripper. To estimate the pose of the tube in the tool-camera frame, we suggest an online sampling-based approach using image gradients in a coarse-to-fine framework. In order to reduce the search space, we employ a biased sampling strategy based on the target shape and projective geometry, and learn promising sampling regions on the fly. Our experiments demonstrate the effectiveness and efficiency of our strategy toward precise Martian sample retrieval.