3D video stabilization for humanoid eyes using vision and inertial sensors inspired by human VOR

This paper proposes a new 3D video stabilization for humanoid eyes using vision and inertial sensors. Most existing video stabilization methods based on 2D motion model work well where scenes are far from the camera or can be modeled as a plane. But these 2D video stabilizations may fail to stabilize unstable videos because a humanoid robot operates in the real 3D world. And so far 3D video stabilization approaches cannot be directly applied to humanoid robot eyes because they use an off-line optimization method. In order to solve above problems, we propose 3D video stabilization based on a spatially-varying warping technique that uses sparse reconstructed 3D data. And we integrate the proposed 3D video stabilization into a hardware stabilization device for humanoid eyes inspired by human VOR in order to compensate large and rapid unwanted motions. The experimental results showed that the proposed 3D video stabilization system can be a potential candidate for humanoid eyes.