Continuous stereo camera calibration in urban scenarios

The quality of a 3D reconstruction obtained from stereo images depends on the accuracy of the stereo camera calibration. Continuous online camera calibration is key to enable a long maintenance-free operation of autonomous vehicles. Conventional algorithms for online calibration usually assume a static world. This, however, is often violated in urban scenarios. In this paper an algorithm is presented that determines the twelve degrees-of-freedom (12-DoF) extrinsic calibration of a stereo camera system in dynamic, urban scenarios. An Extended Kalman Filter (EKF) continuously estimates the extrinsic stereo camera calibration by tracking the position of salient points in 3D space that are observed in both cameras. The EKF predicts the 3D position of tracked points based on the estimated calibration and the vehicle's motion, which is estimated using an inertial navigation system (INS) and odometry sensors. However, only static points can be reliable predicted. A convolutional neural network (CNN) is applied to segment camera images on a pixel level. These segmented images are further converted to binary images labeling static and potentially dynamic pixels. Only salient points which are labeled as static are retained for estimation of the stereo camera calibration. While the stereo camera's extrinsic parameters are only observable under transformation and two independent rotations of the vehicle, we present a robust filter update scheme, that enables estimation of the 12-DoF extrinsic stereo camera calibration even in the absence of significant rotations. Test on urban roads show that minor road imperfections are sufficient to estimate all 12-DoF extrinsic parameters over time.