Relative position estimation for manipulation tasks by fusing vision and inertial measurements

This paper proposes a position estimator that fuses monocular vision with accelerometer and gyro measurements to generate a 6-DOF position estimate between a free-floating underwater vehicle and a stationary object. Camera motion is unknown and must be estimated together with relative position. The system fuses inertial rate sensors to reduce the amount of information that needs to be extracted from the vision system. The result is a system that is simpler and more robust than a vision-only solution. However, the use of inertial rate sensors introduces several issues. The rate measurements are subject to biases, which need to be estimated to prevent the accumulation of unbounded drift when the measurements are integrated. This problem is nonlinear, which presents several challenges in the estimator design. This paper presents some results from initial experiments with a fixed-base manipulator. So far, we have implemented a simplified estimator for relative position when camera motion is known. The estimator is part of the closed-loop control of the manipulator. With this system, we have demonstrated a simple autonomous manipulation task.