Continuous extrinsic online calibration for stereo cameras

Accurate stereo camera calibration is crucial for 3D reconstruction from stereo images. In this paper, we propose an algorithm for continuous online recalibration of all extrinsic parameters of a stereo camera, which is rigidly mounted on an autonomous vehicle. The algorithm estimates the six degrees-of-freedom (6-DoF) of the transformation from the vehicle coordinate system to the coordinate system of the stereo camera and at the same time the relative 6-DoF transformation between the two camera sensors. Salient points in the environment that are observed by both cameras are tracked over time in 3D space. An Unscented Kalman Filter (UKF) is applied to recursively estimate the extrinsic stereo camera calibration and the 3D position of all observed points. The projections of the points and the measured vehicle motion, which is estimated using an inertial measurement unit (IMU), are given as input. The observability of the stereo camera calibration states is analyzed to identify critical vehicle motion sequences. Results with real world data show that the algorithm is capable of continuously estimating the stereo camera calibrations in spite of large initial errors and varying extrinsic parameters.