Three-dimensional visual tracking and pose estimation in Scanning Electron Microscopes

Visual tracking and estimation of the 3D posture of a micro/nano-object is a key issue in the development of automated manipulation tasks using the visual feedback. The 3D posture of the micro-object is estimated based on a template matching algorithm. Nevertheless, a key challenge for visual tracking in a scanning electron microscope (SEM) is the difficulty to observe the motion along the depth direction. In this paper, we propose a template-based hybrid visual tracking scheme that uses luminance information to estimate the object displacement on x−y plane and uses defocus information to estimate object depth. This approach is experimentally validated on 4-DoF motion of a sample in a SEM.