The Extrinsic Calibration of Area-Scan Camera and 2D Laser Rangefinder (LRF) Using Checkerboard Trihedron

The combination of area-scan camera and 2D laser rangefinder (LRF) can capture both textural and geometrical information from a scene at the same time, and has been widely used in various fields. Due to the differences of the installation position and acquisition mode, calibrating the extrinsic parameters, including the rotation and translation, of two sensors is necessary for fusing the camera image and LRF data. In this paper, a simple and flexible extrinsic calibration method is proposed by only acquiring a checkerboard trihedron once. Using checkerboard trihedron as a mobile referenced control field, the proposed method includes three steps to calibrate the extrinsic parameters. First, the rotation and translation between the trihedron and LRF are solved with a simplified perspective-three-point (P3P) solution; Second, using the collinear equation of checkerboard corners and their pixels in the image, the camera is calibrated with respect to the trihedron; Third, combining the last two steps, the rotation and translation parameters between the camera and LRF are finally calibrated with the intermediate referenced trihedron. After a lot of simulation and real experiments, the proposed method has been demonstrated to have the advantages of simple operation, strong robustness and high accuracy in real experiment.


I. INTRODUCTION
The combined use of area-scan camera and 2D laser rangefinder (LRF) is widespread in numerous fields, such as robotics [1], [2], object detection [3], 3D reconstruction [4], autonomous vehicles [5], [6] and localization and mapping [7]. This combination is not only low-cost, but also can obtain rich visual information like color and texture in the scene, as well as the direct geometric structure information like coordinate and distance. But due to the difference of the acquisition mode, installation position, coordinate system between camera image and LRF range data, it is difficult to fuse the two kinds of data of the camera and LRF. To overcome the difficulty, the first and crucial problem is to calibrate the extrinsic parameters between them, in other words, to acquire the rotation and translation matrix between the camera coordinate system and LRF coordinate system. The associate editor coordinating the review of this manuscript and approving it for publication was Md. Moinul Hossain .
The pivot for calibrating the camera and LRF is to find the correspondence between image pixels and LRF points, and build the congruent relationship between them for the purpose of solving the rotation and translation between the sensors. Traditionally, the 3D LRFs [8] and multiline LRFs [9] can acquire the depth information and the intensity information at the same time, so it's easy to find the matching between the image and laser data, and establish the corresponding relationship for extrinsic calibration, such as Zhao et al. [10] and Sui and Wang [11]. Sui's method needs to customize a special calibration object which is a 3D marker with four circles and a checkerboard. Scaramuzza et al. [12] propose a theory which uses image feature points and laser points to build perspective-n-points (PnP) for calibrating. This method needs to use visible laser reflection value for confirming the laser points, and select image feature points and corresponding laser points by manual operator. However, the 3D and multiline LRFs are very expensive and cubersome for an integrated system of multiple sensors. In contrast, 2D LRF shows superiority in low price, so it becomes a better choice in practical applications. But 2D LRF only records the range information, without visible information of echo strength. The 2D LRF points correspond to only a line in the image, and has low spatial resolution. Therefore, it's hard to find the corresponding LRF points in the visual image accurately, and the calibration between camera and 2D LRF becomes more intractable.
As for the extrinsic calibration of camera and 2D LRF, many scholars have done lots of research. In consideration that laser spectrum is out range of visible light spectrum, some researchers use infrared camera to build the correspondence between visible laser points and image pixels, although this is a direct method, the required machines are mostly expensive, and introduce redundant relative relationship between infrared camera, area-scan camera and 2D LRF, which decrease the accuracy of calibration. At present, the most widespread way is using checkerboard as calibration object for extrinsic calibration. Zhang and Pless [13] firstly raise the method of using laser points locating on the checkerboard as constraint to calibrate, but each posture only provides two sets of constraint condition, so getting the extrinsic calibration at least needs five different postures. Zhou and Deng [14] use the constraint that the intersecting line of checkerboard plane and laser scanning plane is orthogonal to checkerboard normal vector. Zhou primarily uses the algebraic structure of polynomials to solve the rotation matrix first, and then solves the translation matrix, which also needs three different poses for solving the extrinsic parameters. The results of the above three approaches all need L-M [15] optimization algorithm, which need relatively accurate initial value, otherwise will get into local minima. Vasconcelos et al. [16] use three diverse poses of checkerboard data to build perspective-three-points (P3P) [17]- [20] problem, and construct virtual trihedron, finally solve the extrinsic parameters with minimal solution. However,these methods still exist the case of multiple solutions and degeneration problems.
In addition, some researchers utilize checkerboards of various shape as calibration target. Wasielewski and Strauss [21] adopt V-shape calibration plate of which the constraint was made by the intersecting points and intersecting lines during the laser scanning on each side, and follows the point-online principle. Li et al. [22] adopt a similar approach to Wasielewski, the difference between them is that Li uses an isosceles triangle board as calibration target. Kwak et al. [23] also use V-shape board, he not only extracts the intersecting line of two planes, but also extracts the edge line of each plane, the intersecting point of laser lines and the laser points on the plane are extracted virtually to provide three pointon-line constraints. But the precision of intersecting point is inferior, so it needs plenty of experimental data to calibrate. Zhao et al. [24] uses a collapsible V-shape board which can continuously change the intersection angle in order to obtain the laser points on the two sides of the V-shape board for building point-on-plane constraint, and also presents an analytical solution for calibration from a single observation. Due to the equipment defects, there's quite big error between the ideal and the actual intersection angle which affects the result of calibration. Recently, some researchers propose a fast extrinsic calibration method based on three perpendicular cubes, for instance, Gomez-Ojeda et al. [25], Briales and Gonzalez-Jimenez [26], Hu et al. [27] and Fan [28]. Gomez-Ojeda uses three orthogonal planes to construct lineto-plane and point-to-plane constraints. Briales and Hu adopt indoor perpendicular corners to construct orthogonal trigonal pyramid, Briales uses line-to-plane constraint and P3P method, while Hu uses perspective-three-lines (P3L) [29], [30] and P3P to solve the position and posture of the LRF and camera with respect to the corner of walls, finally uses minimal resolution to calibrate the LRF and camera. But this method needs to know the size of the calibration target, if the length of the corner is measured imprecisely or the intersection line in image shows blurry, the final result of calibration will be affected. Fan uses photogrammetric control field to calibrate camera, but control field is immovable and extremely expensive to construct. Also, the walls of control field are fixed, it's uneasy to set the location of camera and 2D LRF and make laser scan plane intersect well with the walls and ground.
Inspired by the methods with three orthogonal planes, this paper designs a simple orthogonal trihedron whose surface is labeled with checkerboards. The size and the coordinate of featured patterns on the trihedron are exactly known. Based on this checkerboard trihedron, this paper proposes a handy and flexible method to calibrate the camera and 2D LRF. The checkerboard trihedron has a mass of obvious point features and plane features, and acts as a mobile control field. Using the trihedron as a mobile referenced control field, the proposed method includes three steps to calibrate the extrinsic parameters. First, the rotation and translation between the trihedron and LRF are solved with a simplified P3P solution; Second, using the collinear equation of checkerboard corners and their pixels in the image, the camera is calibrated with respect to the trihedron; Third, combining the last two steps, the rotation and translation parameters between the camera and LRF are finally calibrated with the intermediate referenced trihedron. The proposed algorithm has the following advantages: 1) Flexibility and simplicity of calibration. The checkerboard trihedron can be set in any place with any gesture. Compared with the previous algorithms which need several sets of data, such as [13], [14] and [16] which need at least three gestures of checkerboard, while the proposed method only requires a set of data to complete the calibration of camera and 2D LRF. Also, the checkerboard trihedron is portable and economical compared with the control field of Fan [28], which is exorbitant and inconvenient. 2) High accuracy of calibration results. The control points of the checkerboard corners are easy to extract and uniformly distributed on the trihedron surface. This algorithm avoids the problem of extracting the fuzzy edge of the trihedron in the image, such as [27], and ensures a higher accuracy of the calibration result. 3) Strong robustness of the calibration results. Because a large number of evenly distributed control points are used to provide redundant observation, the robustness of the camera calibration has been improved. At the same time, the translation and rotation of 2D LRF only depends on the intersecting lines of the laser scanning plane and the trihedron, which become insensitive to noise of individual LRF points.

II. CONFIGURING THE COORDINATE SYSTEMS
As shown in Fig.1, taking one of the vertices of the trihedron as the origin of the coordinate system and the intersecting line of the three vertical planes as X W , Y W , Z W axis to build checkerboard trihedron control field O W − X W Y W Z W . Camera coordinate system O C − X C Y C Z C is established by pinhole imaging model, taking the center of the camera as the origin and the plane parallel to the imaging plane as X C − O C − Y C plane, then according to right-handed coordinate system rule, Z C axis is defined by pointing to the trihedron. LRF coordinate system O L − X L Y L Z L takes the scanning center of laser rangefinder as the origin point, scanning plane as the X L − O L − Y L plane, using right-handed coordinate system rule to set Y L axis. Let the coordinate of arbitrary point P be expressed as P W = (X W , Y W , Z W ) T , P C = (X C , Y C , Z C ) T and P L = (X L , Y L , Z L ) T in the checkerboard trihedron, camera and laser rangefinder coordinate system respectively, the homogeneous coordinate of P in the image be as q c = (u, v, 1) T . The intrinsic parameters of the camera is K , the extrinsic parameters of the camera and LRF with respect to the trihedron coordinate system are {R C , T C } and {R L , T L }, the extrinsic parameters between camera and LRF are defined as {R CL , T CL }. The representation of point P in each coordinate system follows: The relationship between the camera and the LRF can be expressed as follows: Combining (2) to (4), the extrinsic parameters between the camera and LRF are as follows: The extrinsic parameters between the camera and LRF are referred to the common checkerboard trihedron coordinate system. The trihedron acts as the medium for the extrinsic calibration in this paper. The following article will begin with calculating the extrinsic parameters of the LRF and camera with respect to the trihedron.

III. CALIBRATING THE EXTRINSIC PARAMETERS A. LRF CALIBRATION
As shown in Fig.2, when 2D LRF is working, its scanning plane intersects with the three orthogonal planes of the trihedron and forms three intersecting lines l 1 , l 2 , and l 3 , shown in blue, red and green lines. LRF records the angle and distance of laser beam, so the coordinates of laser points with respect to O L − X L Y L Z L system can be easily obtained by trigonometric function, and linear equation of l 1 , l 2 , and l 3 can be fit in O L −X L Y L Z L . According to solid geometry principle, the intersection points of lines that are located on orthogonal planes must be located on the edges of the trihedron (orange line segment in Fig.2). Connecting each intersection points constructs an orthogonal triangular pyramid O W − P 1 P 2 P 3 , the length of O W P 1 , O W P 2 , O W P 3 can be computed through the intersection points. Because the physical length in either LRF coordinate system or trihedron coordinate system keeps unchanged, the trihedron coordinate of intersecting points Q 1 , Q 2 , and Q 3 in Q W − X W Y W Z W system can be obtained at the same time. Points Q i (i = 1, 2, 3), in trihedron coordinate system, are the corresponding points to P i (i = 1, 2, 3) in LRF coordinate system, also located on the axis of trihedron coordinate system. Therefore, three couples of laser points and target points construct the 2D-to-3D corresponding relationship to solve the position and posture of LRF with respect to the trihedron, which can be seen as a simplified P3P problem.
On account of the structure of 2D LRF, the scanning plane can be set as Y L = 0, so the point in the LRF coordinate system can be expressed as Firstly, using π 1 , π 2 , and π 3 , to denote the three planes of the trihedron, the laser points locating on the plane can fit three straight lines l 1 , l 2 , and l 3 . The intersecting points P i (i = 1, 2, 3) of l 1 , l 2 , and l 3 in the LRF coordinate system can be computed as follows: The origin point O W of the coordinate system and the three intersecting points P 1 , P 2 , and P 3 constitute a triangular pyramid perpendicular to three planes, O W is the vertex of the triangular pyramid. So it's a typical P3P problem. The length of the triangle pyramid is: Because three sides are perpendicular to each other, it forms a right triangle, the length of the triangular pyramid can be calculated according to the Pythagorean theorem.
Compared with the traditional P3P problem, (9) uses Pythagorean theorem instead of cosine law, and avoids producing the cross terms of sides, which makes P3P problem greatly simplified. Because it's a fact that the side length of triangular pyramid is a positive quantity, the length of OP 1 , OP 2 , OP 3 can be calculated shows as follows: Due to the corresponding relationship between points P i (i = 1, 2, 3) in LRF coordinate system and points Q i (i = 1, 2, 3) in trihedron coordinate system. Thus, points Q i (i = 1, 2, 3) can be represented as Because P i and Q i (i = 1, 2, 3) are the same points that are represented differently in different coordinate system, therefore, the relationship between LRF coordinate system and trihedron coordinate system is equal to the corresponding relationship between P i and Q i (i = 1, 2, 3). Rotation and translation matrix between the above two coordinate system can be solved according to the principle of P3P. P 1 , P 2 , and P 3 in laser coordinate system can form a new spatial coordinate system P 1 − xyz. Setting P 1 as origin point, and unit direction vector of P 1 P 2 as axis x, what's more, unit direction vector which is the height of edge P 2 P 3 in P 1 P 2 P 3 is set as axis y. z axis is perpendicular to plane Due to equivalent corresponding relationship between P i and Q i (i = 1, 2, 3).The rotation matrix R 2 = x y z of P 1 − xyz and trihedron system O W − X W Y W Z W can be obtained in the same way. With the intermediate coordinate system P 1 − xyz, the rotation of LRF coordinate system with respect to trihedron coordinate system is denoted as: With the result of R L , the translation matrix T L can be easily represented as: Standard P3P algorithm is to find the solution of vector O W P 1 , O W P 2 and O W P 3 . In view of the length and directionality of vector, standard P3P algorithm gets 2 3 = 8 solutions, As shown in Fig.2, according to the distribution of laser data, the coordinate of intersection points P 1 , P 2 and P 3 in laser coordinate system can be computed through (7), and the location of intersection points Q 1 , Q 2 andQ 3 in world coordinate system can be determined. Q 1 is located on the positive axis of X W axis, Q 2 is located on the positive axis of Y W axis, Q 3 is located on the negative axis of Z W axis. The lengths of vector O W Q 1 , O W Q 2 and O W Q 3 . are calculated by (10). Thus, the we can determine the unique coordinate of intersection points The intersection points P 1 , P 2 and P 3 in laser coordinate system are calculated through (7). In this way, we have abandoned the 7 solutions which are not consistent with the actual situation and obtain the single solution by using (11) and (12).

B. CAMERA CALIBRATION
With the common medium of the checkerboard trihedron, the LRF has been referred to the trihedron coordinate system in the previous section, resulting in the extrinsic parameters R L , T L . This section will introduce how to recover the relationship between the camera and trihedron at the imaging moment, that is to say the extrinsic parameters R C , T C between the camera and trihedron coordinate system.
As shown in Fig.3, for the perspective projection of the pinhole imaging model of a digital camera, the image pixel a, target point A and the photocenter of the camera O C are collinear, in other words, they are satisfied the collinear equations as follows: where the intrinsic parameters of the camera are (x 0 , y 0 , f ), (x 0 , y 0 ) represents the principal point's coordinate, f represents the focal length of the camera. In Fig.3, the highprecision control point coordinate is A = (X , Y , Z ), the image pixel coordinate corresponding to A is a = (x, y), which is readily picked from the image.
are computed through the nonlinear combination of the rotation parameters φ, ω, κ and the origin X S , Y S , Z S of the cam- All the undetermined parameters are denoted as for simplicity, so the image point coordinate x, y can be represented as the nonlinear function of , i.e., the f x ( ) and f y ( ) respectively. Giving an initial parameter 0 to substitute into (13), then the initial approximations (x), (y) of the image pixel coordinates x, y are acquired. Using the Taylor series expansion at (x) and (y), ignoring the higher order items above quadratic term, (13) can be transformed into the linear form as follows: where ∂x ∂ and ∂y ∂ represents the first order coefficient Taylor expansion of f x ( ) and f y ( ) at (x) and (y) with respect to the rotation and translation parameters. For a given initial parameter 0 , all the first order coefficients are known. The variables dX S , dY S , dZ S , dφ, dω, dκ represent the change of the rotation and translation parameters compared to the initial ones, which are unknown, i.e., (14) is an equation about variable parameter d .x,ŷ are the newly estimated value of image point a based on (x) and (y) through (14).
From (14), it is easy to find that there're 6 unknown variables. Each control point and the corresponding image pixel can only list two equations, so at least three points are needed to obtain the solution for six unknown variables. But the result calculated with only three control points tends to be accidental and inaccurate. Hence, in order to improve the accuracy of the experiment, it's necessary to pick up n even-distributed points on the checkerboard trihedron image, using numerous control points can decrease the effect of accidental error and get more precise result.n(n ≥ 3) points can list 2n equations. There's only 6 unknown variables, using least square method to solve the overdetermined equation by minimizing the sum of squares of residuals, i.e.
which acquires the minimum value when the partial derivative of d tends to zero, and results in the optimum variation of rotation and translation parameters. As for Taylor expansion only keeps at first order in the process of linearization, which omits the high-order items.
Also, the initial values 0 of the parameters are rough, so it's important to amend the result iteratively, in other words, iterative computation. The change of the rotation and translation parameters d and the approximation of them approx calculated in the last iteration are added as the newly initial value for the next iteration. Repeating the iteration until the three rotation parameters φ, ω, κ are all less than 0.1 and then getting the extrinsic parameters R C , T C of the camera coordinate system with respect to trihedron coordinate system. where Compared with the P3L method which uses corner of the wall for calibration, this method is not affected by the vague and poor contrast of the corner image. Through a mass of uniformly distributed control points, legible image pixels and iterative computations, the high accuracy of the extrinsic parameters is guaranteed.

IV. EXPERIMENTAL RESULTS AND ANALYSIS
Both simulation experiment and real experiment are taken in this section to evaluate the performance of the proposed algorithm. Because there always exist various errors in every observation, it's hard to obtain the ground truth of the extrinsic parameters between the camera and laser rangefinder, the simulation experiments are required for testing at the beginning, followed by the experiments in the real world.

A. SIMULATION EXPERIMENT
In order to verify the calibration result of the algorithm, synthetic data with different degrees of noise is used to evaluate its performance. In the simulation experiment, according to the real equipment used in the real experiment, we chose the same device parameters for the camera and 2D LRF. Set the camera parameters as: focal length is 3.2mm, pixel size is 6.7µm, image size is 4608 × 3456 pixel, principal point is the center of image, and the distortion coefficient is 0 for simplicity. Set the laser rangefinder parameters as: angular resolution is 0.25 • , scanning ranges from −45 • to 225 • . Set checkerboard trihedron parameters as: each side of the trihedron has 10 × 10 checkers with the side length of 50mm, and the length of three mutually orthogonal edges is 50cm. The ground-truth extrinsic parameters between the camera and LRF coordinate system are set as followings:  As for knowing the true value of the calibration between the camera and LRF, the errors of rotation and translation matrix are used for evaluating the accuracy of the proposed algorithm. As shown in (20), E R , E T respectively represent the error of rotation and translation matrix. If they are small in the experimental results, it shows that they are close to the ground truth value, and the accuracy of the calibration results is high. Otherwise, the accuracy is poor. The error calculation formula is as follows: Before the simulation experiment of camera and laser calibration, several simulation experiments of camera calibration are needed to prove the superiority of proposed algorithm. For verifying the performance of camera calibration algorithm, we firstly conduct comparative experiment between proposed algorithm and EPNP algorithm. The simulation data of camera calibration is the image points and object points of checkerboard trihedron. Setting image noise from 1 to 10 pixels to image points and 1000 independent experiments are carried out under each noise level. The ground truth of camera extrinsic parameters are set as follows: The reprojection error is the error between the projected point and the measurement point on the image. Reprojection error has always been a common evaluation criterion of camera calibration, so the average value of reprojection errors of two methods in different image noise levels are shown in Fig.4.
As shown in Fig.4, the green line reprensents the reprojection error of proposed algorithm and the red line represents the reprojection error of EPNP algorithm. With the growing of image noises, the reprojection errors of both methods grow to some extent. In Fig.4, it can be illustrated that the value of reprojection error and its rising trend of the proposed algorithm is smaller than EPNP algorithm. When the image noise goes to 10 pixels, the reprojection error of the proposed algorithm is 11.48 pixels, while EPNP algorithm goes to VOLUME 8, 2020 12.52 pixels. Through simulation experiments, the proposed algorithm performs better than EPNP algorithm.
The proposed algorithm needs 3 mutually perpendicular planes to calibrate the LRF and needs one or more checkerboards to calibrate area-scan camera. The calibration mode can be simplified into two cases which shown in Fig.5. As shown in Fig.5(a), the first case is that using two sides boards without checkerboards and one side with checkerboard. The second case is that using three sides boards with checkerboards, as shown in Fig.5(b). As for the LRF calibration in each case keeps the same, only the camera calibration is evaluated for each case. In order to illustrate the precision of each case, we add image noise from 1 to 10 pixels and 1000 independent experiments are carried out under each noise level, while reprojection error works as the evaluation criterion. The simulation experiments results are shown in Fig.6.
As shown in Fig.6, the red line represents the reprojection errors of the first-case camera calibration and the green line represents the reprojection errors of the second-case camera calibration. It can be seen that the value of reprojection error and its rising tendency in the first case is smaller than the second case under all image noise levels. Thus, the plane of checkerboard in Fig.5(a) do provide constraints in camera calibration, while the other two sideboards with checkerboards can also contribute to camera calibration by providing more control points in three-dimensional space, which increase the diversity of world points and improve the accuracy of extrinsic calibration of area-camera.
For verifying the algorithm adaptation in various degrees of noise, in the simulation experiment, the image pixel locations and the laser points are respectively added with some gaussian noise of the zero mean and various deviations, to simulate the different noise levels in the real situation. Noise of the LRF range data is added to the distance from the LRF to the trihedron, and the pixel location of the checkerboard corner in the image is offset by the noise to various degrees.
Firstly, set the LRF noise as gaussian noise with a fixed standard deviation equal to 6mm, the standard deviation of image noise ranges from 1 to 10 pixels,and 1000 independent experiments are carried out under each noise level, the proposed algorithm is used to calculate the extrinsic parameters, the resultant error of the parameters is shown in Fig.7 and Fig.8. Moreover, set the image noise in the experiment as gaussian noise with a fixed standard deviation equal to 0.5 pixel, and the standard deviation of LRF noise is from 2mm to 20mm, and for different levels of noise, 1000 independent experiments are carried out under each noise level. Meanwhile, the experimental result of the state-of-the-art methods by Hu [27] and Fan [28] are also plotted as follows for a comparative analysis. But the simulation scene of Fan's method relies on the control field which is different from the proposed method and Hu's method, its simulation result under the same noise level is referred directly here while evaluating the result of simulation experiment. The input laser data are same in both proposed method and Hu's method, while the input data of camera calibration is quite different. Unlike Hu's method, which works on the three edges of the trihedron, the proposed method takes advantage of many control points on the three planes of the trihedron. Fig.7 and Fig.8 plot the experimental results of the proposed algorithm, the state-of-the-art algorithm of Hu's and algorithm of Fan's. The red line represents the errors of  the proposed algorithm at different noise level, green line represents the errors of Hu's algorithm at different noise level and gray line represents the errors of Fan' s algorithm at different noise level. x axis represents the level of noise, image noise is measured in pixels and laser noise in millimeters, y axis represents the error magnitude, the error of rotation matrix is measured in degree and translation matrix in millimeters. As Fig.7 shows, when small image noise is added, all methods can obtain high-precision results, which the mean of rotation errors are below 0.02 • and translation errors are below 30mm. But when the laser noise is fixed, with the image noise increasing from 1 to 10 pixel, the mean errors of rotation increase form 0.0121 • to 0.0126 • , Fan's mean errors of rotation increase from 0.0016 • to 0.0026 • , while state-art-of Hu's mean of rotation error increase from 0.025 • to 0.094 • . The mean errors of translation increase from 10.436mm to 11.101mm, Fan's mean errors of translation increase from 5.534mm to 6.668mm, while state-artof Hu's mean of translation error increase from 37.509mm, to 153.510mm. So it's simple to figure out that, with the image noise increase from 1 to 10 pixel, the error of the proposed algorithm and Fan's algorithm grow slowly and changes steadily and the error of Hu's algorithm grows relatively faster. As Fig.8 shows that the mean of rotation error increase from 0.0021 • to 0.021 • , Fan's mean errors of rotation increase from 0.0007 • to 0.0045 • , while stateart-of Hu's mean of rotation error increase from 0.008 • to 0.025 • . The mean of translation error increase from 2.935mm to 18.152mm, Fan's mean errors of translation increase from 1.764mm to 10.591mm, while state-art-of Hu's mean of translation error increase from 14.338mm to 26.802mm. It's clearly to figure out when image noise is fixed, the larger the standard deviation of LRF noise, the large the errors of all methods extrinsic calibration. As for the control points are even-distributed in photogrammetric control filed, so the imaging points of Fan's are more equally distributed in image, while the scale of checkerboard trihedron is much smaller than control field, the corner points of checkerboard trihedron are uneven-distributed than control field,in this way, Fan's method performs a little better than the proposed method. But the proposed algorithm performs much better than Hu's. This is because the proposed method uses checkerboard trihedron as a control field tool and calculates the position and posture in a rigorous mathematical model, which makes a smaller experimental error and higher precision of calibration result. As for the proposed method, when image noise achieves 10 pixels, the error of rotation matrix is less than 0.02 • and the error of translation matrix is not exceeding 30mm. When LRF noise achieves 20mm, the error of rotation matrix is less than 0.025 • and the error of translation matrix is no more than 35mm. In conclusion, the proposed algorithm has a high-precision calibration result and strong robustness to noise.
To observe the accuracy of simulation result qualitatively, the laser data of each group are mapped to the camera coordinate system by the calculated extrinsic parameters, that is to say the LRF points are projected onto the image, and compare with the projection using the ground-truth extrinsic parameters. Because of the different simulation scene and different calibration target used in Fan's method, only the proposed method and Hu's method are observed by projecting onto image.
As shown in Fig.9 and Fig.10, X i and Y i represent the coordinate axis of image, the red, blue and green points represent the projection of LRF points onto the image plane VOLUME 8, 2020  with the ground-truth parameters, the calibrated parameters of the proposed algorithm and Hu's algorithm. Because the rigid transformation between different coordinate systems brings no distortion and deformation, the straight lines and intersecting points of the LRF data obtained by the two methods are still preserved and invariant. When the standard deviations of LRF noise and image noise are small, the proposed algorithm and Hu's algorithm are quite close to the ground truth in that they almost overlap together, as shown in Fig.9(a) and Fig.10(a). With the increase of LRF noise and image noise, the calibrated results of extrinsic parameters include errors, the projection of two methods are distorted and offset, compared with that of the ground truth, and the intersecting point of LRF profile is gradually offset far away from the ground truth. But in Fig. 9(c) and Fig.10(c), it's easy to find out the projection result of the proposed algorithm is more accurate than Hu's, the degree of distortion and deviation is smaller.The position of the intersecting point has not changed much, especially for the fixed image noise, so it can prove the proposed algorithm can obtain an accurate and robust result of extrinsic calibration.
Considering the accuracy in the production process of trihedron, the two planes of checkerboard may not be completely perpendicular. Taking two planes π 2 , π 3 as example, we add angle noises from 0.25 • to 1 • and 1000 experiments are carried out under each noise level. The precision of calibration results are evaluated by using (20), the mean errors of rotation and translation error is shown in Fig.11, where the image and laser noise in the experiments are fixed to 0.5 pixel and 6mm respectively.
As shown in Fig.11, blue rectangle represents the error of calibration rotation and translation error. x axis represents the degree of angle error of being perpendicular between the two perpendicular planes(π 2 − π 3 ). y axis represents the error of rotation and translation matrix. The rotation error and translation error with no angle error of two perpendicular planes are 0.0126 • and 10.970mm. With the growing of angle noise, the rotation error and translation error gradually increase but in a moderate trend. When the angle noise increase from 0.25 • to 1 • , the rotation error increase slightly from 0.0128 • to 0.0137 • and the translation error increase from 11.221mm to 11.378mm, which demonstrates that the production accuracy of perpendicular planes has little impact on the proposed LRF calibration.

B. REAL EXPERIMENT
From the above large number of simulation experiments, it can be seen that the proposed algorithm can obtain the extrinsic parameters with high accuracy at different levels of image and LRF noise. In order to test that the proposed algorithm also performs well in reality, the following experiments are carried out.
The equipment, which is used in the real experiment shown in Fig.12(a), is composed by six camera and three laser rangefinders, and can obtain image and laser data at the same time. The No.1 Camera and No.2 Laser rangefinder are used,  the resolution of camera is 4608 × 3456 pixels, the focal length is 3.2mm. The scanning scope of the LRF is 270 • , and the resolution of scanning angle is 0.25 • . A total of 1081 LRF points is obtained at one scanning. The checkerboard trihedron is shown in Fig.12(b), with a size of 0.5m×0.5m×0.5m, and a 10 × 10 checkerboard on each side. The size of each cell is 50mm × 50mm. The specific parameters are shown in Table.1 As shown in Fig.13(a), in the real experiment, both the camera and LRF are placed toward the trihedron, making the LRF scan two surfaces of the trihedron and the ground, and the camera capture the entire cube clearly. Because the projection from the LRF to camera coordinate  satisfies the rigid transformation, the laser line is still straight in the image, and the intersecting point of the laser line is still an intersecting point in the image because it is located on both lines. Based on such characteristics, the LRF points are transformed into image pixels with the extrinsic parameters to verify the precision of the extrinsic calibration.
The image and laser points obtained during the real experiment is as follows: Fig.13(a) represents the image data, in which many corner points of the checkerboard trihedron are readily detected, thus providing many control points for the camera calibration. Also, Fig.13(a) shows LRF scanning plane, Fig.13(b) represents the LRF range data, laser points are located on the laser system, among which the ''V-shaped'' corner is the laser data that intersects the two surfaces of the trihedron. The green, red, and blue part represents the intersection line between the LRF scanning plane and trihedron left plane, trihedron right plane, and the ground plane respectively.
By capturing the trihedron once, the calibration result of the LRF and camera with respect to the trihedron is obtained in (22) and (23).
Fig.14 is the projection results of the LRF points onto the image using the extrinsic parameters between the camera and LRF. For the rigid transformation between two coordinate systems, there makes no change of the shape of an object. After the perspective projection of the pinhole imaging model of digital camera, a line in the camera coordinate system remains linear in the captured image. The rectangle in Fig.14 shows that the projection of laser points that linearly intersect the surface planes satisfies the linear feature, and sticks closely to the surface of the trihedron in the image. Laser beam is deflected with the shape of the trihedron, which makes many segments of laser points. All the segments are also closely located on the plane of the trihedron surface.
Besides the qualitative evaluation of the projection results, quantitative measure is also proposed to evaluate the experimental results. It's incapable to obtain the ground-truth extrinsic parameters between the camera and LRF in the integrated device, so the experimental result can't be compared with a true value. Also, the LRF points can't be directly captured by the spectrum of area-scan camera, it is hard to perceive the result of calibration. Instead, a point-to-edge (P2E) distance is proposed in this paper, based on the relationship between the intersecting point of LRF segments and the intersecting line of the underlying planes.
As shown in Fig.15, black lines represent the linear laser segments modulated by the trihedron, π 1 , π 2 are their underlying planes, l is the intersecting line of plane π 1 , π 2 , P 1 , P 2 are the intersecting points of the fitted laser lines. When there's no noise in the camera and LRF and extrinsic parameters are calibrated exactly, the intersection point shows as P 1 , which just right locates on the edge of checkerboard trihedron. But in real experiment, noise and inaccurate calibration cause the intersecting point drifting away from the edge, P 2 is no more on the edge of checkerboard.So if the higher accuracy of extrinsic calibration is obtained, the smaller the reprojection error of LRF points onto the image is, resulting in smaller P2E distance and better fusion of two sensors. Hence, the distance of point P 2 to edge works as the evaluation measure of real experiment.
Back to Fig.13, the red point in the yellow circle is the projection of the intersecting point of the laser lines in the LRF coordinate system. It can be seen from the enlarged details that the intersection point is basically located at the edge of the checkerboard trihedron in the image. This proves the accuracy of the proposed extrinsic calibration.
Further, in order to quantitatively exploit the accuracy of the calibration results in real scenarios, using camera-LRF integrated device scanning the actual scene of various shapes on the wall, the laser points are projected onto the image. P2E distance is used for evaluating R CL , T CL obtained above. The integrated equipment is set about 2.7m away from the walls and shot the walls almost vertically. As shown in Fig.16, blue line represents the fitting line of LRF points projected onto the image, blue point represents the individual projection of LRF point, while green line represents the edges of the cube and the yellow point represents the intersection of laser line and shape surfaces. The intersecting points are magnified in the bottom of Fig.16. It can be seen that the total laser data is quite fit on the wall and the shapes, the intersecting points of LRF segments are located closely to the edge of the shapes after the projection using the calibrated extrinsic parameters in (24).
The P2E distance of the intersecting points is listed in Table 2, the maximum value is about 3 pixels and the average value is 1.686 pixels, which demonstrates the accuracy of the proposed calibration between the camera and LRF. Meanwhile, the extrinsic parameters solved by Hu's The projection of LRF points onto the same image using Hu's and Fan's parameters are also shown in Fig.17, where the blue points, green points and red points respectively represent the projection of laser data with the proposed method, Hu's method and Fan's method. Fig.17 shows that the projection result of the proposed method and Fan's method fit well with the wall and the shapes.
Usually the noise of LRF data varies with the laser detective distance, the longer distance is, the bigger noise of LRF data are. In the meantime, the image features could be better captured at the indoor with brighter illumination than outdoor poor lighting. Thus, to better demonstrate the performance, especially the robustness of the proposed method, we add many real experiments at both the indoor and outdoor scenarios, and with different measuring distances. Because the result of Hu's algorithm produces such offset, the P2E   disatance is not evaluated. The projected LRF points aren't accurately fallen inside the shapes on the wall in the image, although the LRF points do intersect the shapes in the LRF coordinate system. Fig.18 represents the LRF points projection on to the image of indoor scenario, Fig.19 represents the LRF points' projection onto the image of outdoor scenario. As shown in Fig.18 and Fig.19, the blue point represents the projection of LRF point using the proposed algorithm, the blue line represents the fitting line of projection LRF points based on proposed algorithm, while light blue point represents the projection of LRF point using Hu's algorithm, the light blue line represents the fitting line of projection LRF points based on Hu's algorithm, and yellow point represents of the intersection point of fitting laser line, green line represents the edge of corner in the scenarios. The integrated equipment is set 3.2m away in Fig.18, and the setting distance of Fig.19 is 6.5m.
As shown in Fig.18 and Fig.19, the projection of LRF points using the proposed algorithm satisfies the linear feature and sticks closely to the surface of the structure in the image. In the meantime, the projection of intersection points using the proposed algorithm are more closer to the edge of corner than using Hu's algorithm. As for P2E distance is used for evaluating extrinsic parameters of camera and LRF, the average P2E distances of the proposed and Hu's algorithm in Fig.18 are listed in Table.3, the proposed algorithm's maximum value is about 2.7 pixels and the average P2E distance is 1.962 pixels, while the Hu's algorithm's maximum value is about 13.2 pixels and the average P2E distance is about 12 pixel. As for the Fig.19, the P2E distance of algorithm is 5.493 pixels, while the P2E distance of Hu's algorithm is 27.981 pixels. Although the accuracy of the proposed algorithm decreases with the distance, it can be figured out that the errors of the proposed algorithm varies less with distance than Hu's algorithm and still get a high accuracy in different scenarios.

V. CONCLUSION
This paper proposes a simple and flexible calibration method between the area-scan camera and 2D laser rangefinder based on a checkerboard trihedron. It only needs to collect the checkerboard trihedron once, which simplifies the calibration process and speeds up the calibration. Additionally, the checkerboard trihedron is very easy to make and operate. After a large number of simulation and real experiments, the accuracy of the proposed extrinsic calibration has been proved effective in practical application both qualitatively and quantitatively.