IMU-camera self-calibration using planar mirror reflection

In this paper, we first look at the problem of estimating the transformation between an inertial measurement unit (IMU) and a calibrated camera, based on images of planar mirror reflections (IPMR) of arbitrary feature points with unknown positions. Assuming that only the reflection of the feature points are observable by the camera, the IMU-camera calibration parameters and the position of the feature points in the camera frame are estimated using the Sigma-Point Kalman filter framework. In the next step, we consider the case of estimating varying camera intrinsic parameters using the estimated static parameters from the previous stage. Therefore, the estimated parameters are used as initial values in the state space model of the system to estimate the camera intrinsic parameters together with the rest of the parameters. The proposed method does not rely on using a fixed calibration pattern whose feature points' positions are known relative to the navigation frame. Additionally, the motion of the camera, which is mounted on the IMU, is not limited to be planar with respect to the mirror. Instead, the reflection of the feature points with unknown positions in the camera body frame are tracked over time. Simulation results show subcentimeter and subdegree accuracy for both IMU-camera translation and rotation parameters as well as submillimeter and subpixel accuracy for the position of the feature points and camera intrinsic parameters, respectively.