Extrinsic calibration of a fisheye camera and inertial measurement unit

Due to its robust performance and wide potential application, the fusion of visual and inertial sensors for motion estimation has attracted significant attention recently. For this fusion, extrinsic calibration is a prerequisite. But most existing methods need a special configuration in which the camera can observe the inertial sensor. In this paper we present an extrinsic calibration method for visual and inertial sensors in a general configuration in which the visual and inertial sensors can not observe each other. The proposed method introduces a total station as a reference. The total station can be adjusted to be parallel to the ground plane to which the inertial sensor can provide the rotation. Then the rotation between the fisheye camera and the total station is deduced via a homography. Finally the rotation between the fisheye camera and the inertial measurement unit can be obtained. We implement the proposed method on our lunar rover prototype. Experiment results show the validity of the proposed method.