Accurately calibrate kinect sensor using indoor control field

Microsoft Kinect is a new and robust 3D camera, which can be used for indoor scene and 3D model reconstruction. It contains an infrared projector, an infrared camera and a RGB camera, color and depth map can be captured in one scene at the same time and run at a speed of 30 frames per second. As a handhold device, Kinect is low-cost and portable compared with lidar, but its accuracy is low. The internal parameters of both infrared camera and RGB camera, as well as their relative pose, are pre-calibrated in factory, however these average values can't meet the need of high-precision applications and parameters vary from device to device. So if we want to improve the precision of Kinect, the calibration should be done at first. In this article, we attempt to use indoor control field to calibrate Kinect sensor, and get the accurate internal parameters of both cameras and their relative pose. Different from some compute vision methods, the 3d coordinate of control points should be measured in our coordinated system and one image is enough fine to compute the internal and external parameters. As Kinect can't get the IR stream and color stream simultaneously, we get the depth and color image at first, then cover the infrared projector and get the IR image, while an infrared compensation lamp is used to make the IR map clear. We separately detect the control points in the color and IR image for getting their corresponding image points, then gain their distance values from depth image based-on IR image points. Our algorithm contains four steps: 1) Initializing the internal parameters of infrared camera and RGB camera with Zhang's method. For RGB camera, collinearity equation is applied to establish the relationship between control points and their RGB image points, then estimating external parameters and refining internal parameters based-on least square adjustment. For infrared camera, transform IR image points to Kinect 3D coordinate with their distance value and initialized internal parameters to establish the point-to-point correspondence with control points, then calculate external parameters and refining internal parameters using iterative closest points method. Furthermore, we assume a model to improve the precision of distance value, in which three additional parameters should be estimated. 2)Projecting the control points to IR and RGB image coordinate separately which consider as ideal points, and regard the corresponding image points as real points, we can estimate the distortion parameters of infrared and RGB camera. 3) Iterating 1)2) steps until it is convergence. 4) Using the external parameters of both cameras to calculate their relative pose. In our experiment, 36 control points are measured, part of them (about 20) can be seen in one image. We usually use 15 points of them to estimate parameters and others consider as check points. The results show that our method can get high precision calibration parameters which projection mean square error lower than 0.5 pixels, and the mean square error of transforming depth image points to RGB image points lower than 1.0 pixels.