Method for Horizontal Alignment Deviation Measurement Using Binocular Camera Without Common Target

In the application of using binocular camera to measure the horizontal alignment deviation, due to the limited field of view of the camera, it is difficult for each camera to obtain all the target points to determine the position of the parts. To this end, a binocular position measurement model, where each optical camera corresponds to one cooperative target point, is established to determine the horizontal alignment deviation between the upper and lower parts. To overcome the problem of insufficient information in calculating the horizontal alignment deviation, the measurement data of a biaxial inclinometer and laser rangefinder are added. The Monte Carlo method is used to determine the error level for any position and attitude in the measurement range based on past work experience. The high accuracy of the position measurement method (the overall error does not exceed 2 mm) for a binocular camera without a common target meets the actual application requirements. This conclusion is verified by the experimental results.


I. INTRODUCTION
Automation and digitalization in the industrial development promote the advancement of the automatic alignment technology, and position measurements are the key technology for realizing automatic alignment. The vision-based method is one of the most important position measurement methods. The existing visual position measurement methods can be mainly classified into monocular vision, binocular vision, and multi-eye vision methods [1]- [3]. The current position solution method for monocular vision is relatively mature; however, in the practical application of monocular cameras, the data amount is often limited owing to the limited field of view and shooting angle. Owing to the reduced camera prices and improved computer performance, the method in which two or more cameras are used to expand the field of view and improve the measurement accuracy has been widely researched and applied [4]- [6]. The measurement method The associate editor coordinating the review of this manuscript and approving it for publication was Datong Liu . based on many cameras is mostly used for large-scale areas such as football fields and outdoor human motion capture systems because it requires complete calibration between the two cameras. By contrast, the binocular camera has been widely applied for small-scale measurements because of its relatively simple system structure [7], [8]. Most current position measurement methods based on binocular vision focus on the binocular stereo camera. The methods use two cameras with fixed geometric relationship to obtain the image information of the measured object from different perspectives; the matching features of the same visual information of different cameras are used to complete the position measurement; thus, the two cameras need overlapping fields of view [9]- [11].The target point located in the overlapping field of view of the two cameras is called the ''common target''. However, under special circumstances, the two cameras will not be able to obtain overlapping fields of view or targets owing to the occlusion of the object or limitation of the field of view. Thus, a binocular camera position measurement method without a common target is required.
The current solution methods for the position measurement problem with multiple cameras and without a common target can be mainly classified into two categories: the first category applies the idea of iterative algorithms in the visual measurement to establish a generalized camera model; the information captured by different cameras is converted into a unified generalized camera coordinate system for representation [12]. By introducing a nonlinear cost function, such as the reprojection error and collinear error of spatial points, the Gauss-Newton, Levenberg-Marquardt(LM) and other optimization algorithms are used to solve the problem iteratively [13], [14]. The other category comprises non-iterative methods, which uses the mapping relationship between the cooperation targets and image projection points or the constraint relationship between the cooperation targets to establish the linear equations [15]- [18]. By using the method with the parameterized rotation matrix, the position problem is transformed into an optimization problem of the rotation parameters; subsequently, the polynomial equations of the rotation parameters are solved to obtain the closed-form solution to calculate the position [19]- [22]. However, for the method using the points feature of cooperative targets, no matter whether it is an iterative method or a non-iterative method, the n ≥ 3 available target points are required; when the number of target points is below three, the existing measurement method fails, and other solutions must be sought [23]- [25].
This article presents a scheme for measuring the horizontal alignment deviation of an automatic alignment system by using a binocular camera without a common target. The main contributions are as follows: 1) Different from most current binocular vision measurement systems, the two optical cameras in the proposed system have no common target point. The relationship between the two calibrated camera coordinate systems is used to establish the relationship between the imaging equations of different cameras and to convert the information captured by the two cameras into a unified camera coordinate system for representation. The scheme overcomes the limitation of the automatic alignment system, which cannot be configured with traditional binocular stereo cameras for measurements owing to the limited field of view. 2) The proposed scheme is different from Perspectiven-Point algorithms, which require at least three cooperation target points. By adding the measurement data of the biaxial inclinometer and laser rangefinder, the horizontal alignment deviation between the upper and lower parts can be solved with only two target points. The rest of this article is organized as follows: Section 2 introduces the automatic alignment measurement system, and Section 3 presents the binocular camera position measurement model for the measurement system. Subsequently, the error sources of the system are analyzed, and the measurement error analysis based on Monte Carlo method is proposed in Section 4. Section 5 presents the simulation and experimental results. Finally, the conclusions are presented in Section 6.

II. MEASUREMENT SYSTEM OVERVIEW
In this study, the binocular cameras are set up as in Fig. 1. The two cameras are located in the optical measuring devices A and B, respectively. The optical measuring devices are used to obtain the image coordinates of the target points and the distance data between the moving part and the fixed part. The biaxial inclinometer is used to measure the horizontal inclination angles of the upper moving part. Based on these, the horizontal alignment deviation of the two parts can be measured. Then we can adjust the moving part to complete the alignment and lower the moving part with gravity. The measuring system consists of two cooperative target points 1 and 2 on the bottom fixed part and a biaxial inclinometer and two optical measuring devices A and B on the moving part. The cooperative targets are made of the Porro prism ( Fig. 2(a)), which can be used in natural light without an additional light source due to its reflective characteristics. The biaxial inclinometer, which consists of high-precision gyroscopes, is installed on the upper moving part to measure the pitch angle β and roll angle γ of the moving part in the horizontal direction ( Fig. 2(b)). Each optical measuring device contains an optical measuring camera and a laser rangefinder, which are shown in Fig. 2(c). There is no common target between the two optical cameras of the two optical measuring devices. In addition, the positional relationships between the two cameras and between the optical camera and laser rangefinder are calibrated [26]- [29].
When there is only a small misalignment between the fixed and moving parts, i.e., the Euler rotation angles between the two parts are α, β, γ ∈ [−1 • , 1 • ], the measured distances in the horizontal and vertical directions are t x , t y ∈ [−100mm, 100mm] and t z ∈ [700mm, 1300mm], respectively. During the measurement, the two cameras without a common target shoot the target points 1 and 2 on the diagonal of the bottom fixed part to obtain the image coordinates of the target point, respectively. The horizontal inclination data of the moving parts measured by the biaxial inclinometer and the distance data of the upper and lower parts measured by the laser rangefinder are used to calculate the horizontal VOLUME 8, 2020  alignment deviations t x and t y of the two parts; subsequently, the moving part is adjusted to complete the alignment. In this process, the measurement error of the system in the horizontal direction must be below 2 mm. Fig. 3 shows the coordinate system of a binocular measurement system without common target. The center of the fixed part is the center of the measurement reference coordinate system O − XYZ , O is the center point of the rack, and P 1 (X 1 , Y 1 , Z 1 ) and P 2 (X 2 , Y 2 , Z 2 ) are the coordinates of the cooperation targets in the reference coordinate system; in addition, Z 1 = Z 2 = 0; O −X Y Z is the upper alignment coordinate system and O is its center; the two camera coordinate system o a − x a y a z a and o b − x b y b z b are shown in Fig. 3. The positional relationship between the two cameras is calibrated, and the internal parameters of the cameras are known; a (u 1 , v 1 ) and b (u 2 , v 2 ) are the coordinates of P 1 and P 2 on the image plane, which corresponds to the CCD discrete array of the photo sensing elements, respectively. The coordinates with the physical units of points P 1 and P 2 in the image plane are denoted by a(x 1 , y 1 ) and b(x 2 , y 2 ), respectively. The relationship between (x i , y i ) and (u i , v i ) is as follows:

III. MODEL OF POSITION MEASUREMENT
where dx and dy are the effective pixel sizes in the x and y directions of the image plane, and (u 0 , v 0 ) represents the image center. Based on the measurement coordinate system in Fig. 3, a mathematical model is established to determine the horizontal deviation between the upper and lower parts. The rotation matrix from the upper alignment coordinate system to the bottom reference coordinate system is established as follows where c stands for cos and s stands for sin; α, β and γ are the Euler angles around the Z -, X -and Y -axes; β is the pitch angle and γ is the roll angle. In addition, the counterclockwise direction is positive, and the translation matrix defined by the three-dimensional coordinate transformation is expressed as t = (t x , t y , t z ) T [30]. Consequently, the rotation matrix from the bottom reference coordinate system to the upper alignment coordinate system can be expressed as Subsequently, the binocular measurement model is established; the transformation relationship between the measurement coordinate systems is shown in Fig. 4. According to Fig. 4, the imaging equation of cooperation target where K imga camA is the homogeneous transformation matrix from the camera A coordinate system to the corresponding image plane, which is related to the camera focal length; must be used to establish the relationship between the imaging equations of the target points 1 and 2. The imaging equation of cooperation target where K imgb camB is the homogeneous transformation matrix from the camera B coordinate system to the corresponding image plane, which is related to the camera focal length; T camB camA is a homogeneous matrix related to Euler angles α ab , β ab , γ ab and translation distances t xab , t yab , t zab between the two camera coordinate systems. To facilitate the solution, the translation t 1 is performed before the rotation R 1 to represent the transformation pathway from the reference frame to the alignment frame, and the transformation relationship between other coordinate systems is expressed by the commonly used expression of rotation before translation. Thus, (3) and (4) can be written in the homogeneous matrix form: where λ 1 and λ 2 are the scale factors; their values are the depths of the target points along the optical axes of the respective cameras; n 11 −n 33 and m 11 −m 33 are functions of the Euler angles α da , β da , γ da and α ab , β ab , γ ab (shown in Fig. 4), respectively. The translation distances on the corresponding axes of the alignment frame to camera A coordinate system and from camera A coordinate system to camera B coordinate system are t xda , t yda , t zda and t xab , t yab , t zab , respectively. Because each camera corresponds to one target point, the camera is equivalent to a monocular camera. Owing to the inherent 3D ambiguity of a single camera [31], the laser rangefinder data are used to obtain the depth information of target points 1 and 2 in the respective camera coordinate systems (i.e., the scale factors λ 1 and λ 2 ). The projections of the measurement coordinate system on O −X Z (green) and O −Y Z (blue) of Fig. 2 are shown in Fig. 5.
In Fig. 5, A and B are the positions of the laser rangefinders in the two optical measuring devices, A and B are the VOLUME 8, 2020  node positions of the two optical camera systems. Moreover, γ and β are the roll and pitch angles of the upper moving part measured by the biaxial inclinometer, respectively; Z A and Z B are the distances measured by the laser rangefinders. The spatial coordinates of A and B in the corresponding camera coordinate system are (x a , y a , z a ) and (x b , y b , z b ), which can be obtained based on the system structure design, respectively. The following relations can be deduced based on Fig. 5: The projection of the measurement coordinate system on Z −AO B (red) is shown in Fig. 6; Z a and Z b are the vertical distances from the optical camera nodes A and B to the bottom plane, respectively; t z1 is the vertical distance between the origin of the alignment frame and the origin of the reference frame, and γ 3 is the angle between the Z and Z axes. According to Fig. 6, Z a and Z b can be written as follows: In the next step, t z1 can be calculated: By using the constraint relationship of the rotation matrix rank, (5) and (6) can be solved, which leads to the following expressions: Based on the Euler angles β and γ measured by the biaxial inclinometer in real time, the scale factors λ 1 and λ 2 can be determined with (12). The focal length of the two cameras is known. The conversion matrix T camB camA between the two camera coordinate systems and the conversion matrix T camA align between the camera A coordinate system and the alignment coordinate system can be obtained by prior calibration. By using the coordinates of the cooperation targets 1, 2 in the reference frame O − XYZ and their corresponding image plane coordinates, the rotation matrix R 1 and translation matrix t 1 from the reference frame to the alignment frame can be determined based on (5) and (6). From t 1 = (t x1 , t y1 , t z1 ) T , R 1 = R and t = −R 1 t 1 , the translation matrix t = (t x , t y , t z ) T from the upper alignment coordinate system to the bottom reference coordinate system can be obtained; subsequently, the horizontal alignment deviations t x and t y between the upper and lower parts can be determined.

IV. ERROR ANALYSIS
To evaluate the accuracy of the proposed measurement method, the horizontal alignment deviation must be measured by considering the system errors. Therefore, the system installation errors, sensor parameters, and measurement errors are considered, and the influence of the error term on the measurement accuracy is studied with the Monte Carlo method [32]- [34].

A. ERROR SOURCE
Based on the literatures and previous related work experience [33], [35], [36], the system error sources are as follows: 1) Error of cooperation target E d : after the positions of the cooperation targets 1 and 2 are fixed, their coordinates with respect to the reference coordinates can be considered ideal; however, the actual target point position and ideal value exhibit a deviation. Although the position errors are fixed after the target points are processed and assembled, the influence of this error cannot be neglected. 2) Error of image location E p : it is expressed as the difference between the pixel coordinates of the image measured by the two optical measuring devices and the true pixel coordinates. The magnitude of the error is related to the image extraction algorithm and lens distortion. 3) Error of camera internal parameters E f : it refers to the difference between the nominal value and the real value of each internal parameter of the cameras. Because the pixel size accuracy of the current industrial camera is relatively high, only the influence of the camera focal length is considered, and the errors caused by other internal parameters of the camera are no longer analyzed. 4) Error of angle measurement E j : it refers to the difference between the pitch angle β about the X axis measured by the biaxial inclinometer and the roll angle γ about the Y axis from the true value.

5)
Error of ranging E c : it refers to the difference between the distance measured by the laser rangefinder on the optical measuring device and the actual distance.

B. MEASUREMENT ERROR ANALYSIS BASED ON MONTE CARLO METHOD
In general, the range of each error term can be obtained based on previous experience or actual measurements; however, the exact magnitude of the instantaneous error cannot be determined owing to its randomness. The error analysis in this article can be qualitatively a problem of finding the maximal absolute value of the measurement error within the allowable error range of the system: where E = E d E p E f E j E c is the error vector of the system, and P = α β γ t x t y t z is the pose of the moving part. In other words, the measurement accuracy of the system is determined by the actual position and errors of the system. Therefore, the characteristics of the random distribution of the Monte Carlo method can be used to analyze the horizontal alignment deviation of the automatic alignment system. Under the condition that the error sources are independent of each other and prior probability information is unavailable, it can be assumed that each error follows a uniform distribution. Subsequently, the Monte Carlo method is used to calculate the horizontal alignment deviation for different positions and errors. In the measurement range, a certain number of system positions and postures are randomly selected, and the system error term is randomly introduced into the generated postures. Thus, the same number of values of α, β, γ , t x , t y , t z , E d , E p , E f , E j and E c are randomly selected as the samples of Monte Carlo algorithm. The selected poses and errors are inserted into the binocular position measurement model to calculate the horizontal alignment deviation of the components. For t x and t y , the greatest error values are selected as the maximal measurement errors. In the next step, the numbers of selected poses and errors are adjusted, and the relationship between the maximal error and the number of samples is analyzed. The detailed steps of the analysis of the horizontal alignment measurement error with the Monte Carlo method are shown in Algorithm 1.

V. SIMULATION AND EXPERIMENT A. SIMULATION
To verify the accuracy of the horizontal alignment deviation measurement method with the binocular camera without a common target, the simulation experiment is designed according to the position measurement model: the error sources are substituted into the simulation measurement model, and the relationship between the maximal measurement error of the model and number of selected samples is analyzed with the Monte Carlo method to determine whether the system measurement errors meet the measurement accuracy requirements. The measurement range of the for j = 1 to 10 n do 3: Select the pose P (j) ∈ S and system error E (j) ∈ W randomly; 4: Calculate the horizontal alignment deviation t x (j) and t y (j);

5:
Calculate measurement error E t x (j) and E t y (j); 6: end if 11: if E t y (j) > E t y (i) then 12: E t y (j) = E t y (i) 13: else 14: E t y (i) = E t y (j) 15: end if 16: end for 17: end for system is α, β, γ ∈ [−1 • , 1 • ], t x , t y ∈ [−100mm, 100mm], t z ∈ [700, 1300mm],. According to previous work experience, the values of each error source are as follows: Fig. 7 shows the relationship between the maximal error and number of samples using the Monte Carlo method; the x-axis represents the increase in the number of samples from 10 1 to 10 9 on a logarithmic scale, and the y-axes present the absolute values of the t x and t y maximal errors in each random sampling. The measurement errors of the horizontal alignments t x and t y are below 2 mm, which meets the measurement accuracy requirements. When the number of selected samples reaches 10 7 , the maximal measurement error of the system begins to stabilize. Fig. 8 presents the model error distribution for 10 7 samples. The absolute value of the maximal error of t x is 1.881 mm, and the mean square error is 0.577 mm. The absolute value of the maximal error of t y is 1.793 mm, and the mean square error is 0.486 mm.  B. EXPERIMENT Fig. 9 presents the binocular position measuring system without common target. The obliquity of the upper moving part in the horizontal direction is measured by the biaxial inclinometer located on the moving part, and the image coordinates of the cooperative targets 1 and 2 are photographed by two optical measuring devices located on the upper part. The radius of the cooperative targets used in the system is 12mm, and the resolution of the biaxial inclinometer is 0.001 • . The display of the biaxial inclinometer and optical measuring device is shown in Fig. 10. The biaxial inclinometer displays the pitch angle β and roll angle γ from top to bottom, while the optical measuring device displays the image coordinates, laser ranging value and system status.
The optical cameras in the optical measurement devices use MT9V032 CMOS digital image sensors, which pixel size is 6µm × 6µm, and the resolution is 400pixel × 400pixel. The focal length of the camera lens is 35mm. The positional relationship between the coordinate systems of the two cameras T camB camA (α ab , β ab , γ ab , t xab , t yab , t zab ) and between the upper alignment coordinate system and camera A coordinate system T camA align (α da , β da , γ da , t xda , t yda , t zda ) are calibrated in advance. The relevant system parameters are shown in Table 1.
The initial state is selected as the reference, and the actual moving distance of the moving part is measured by two theodolites. During the measurement, two cameras located in the optical measuring device take pictures of the respective bottom cooperative target points to obtain the image coordinates of the points. Subsequently, the data measured by the biaxial inclinometer and laser rangefinder in real time and the conversion relationship between the two camera coordinate systems are used to obtain the horizontal alignment deviation between the upper and lower parts as the measured value and to calculate the distance of the part moving along the horizontal direction in each step. The distance is compared with the actual distance measured by the theodolites to calculate the measurement error of the horizontal alignment deviation measurement method.    The results of 100 randomly selected data sets are shown in Fig. 11. The absolute values of maximal error of t x is 1.527 mm, and the mean square error is 0.635 mm; the absolute values of maximal error of t y is 1.539 mm, and the mean square error is 0.702 mm. The FLOPs of our method are 767. In the Intel Core 8 Duo with 1.6 GHz standard computer environment, the average time of 1000 runs of the method is 0.45ms. The statistical information of the error data is shown in Fig. 12. The errors of t x and t y are mainly distributed within ±1mm (which corresponds to 87% and 82%, respectively). In addition, the overall errors do not exceed 2 mm, which meets the actual application requirements and proves the effectiveness of the proposed method.
There are some pose measurement methods which all meet the accuracy index mentioned above, however, their use conditions and characteristics are different. These methods are compared in Table 2. Iterative algorithms, such as SaM, need at least four target points to work out the pose of the object to be measured [12]. However, the amount of computation increases sharply with the number of target points. Non-iterative algorithms, such as EPNP or RPNP, have higher computational efficiency, but require more target points to achieve higher accuracy [20]. In our engineering application, both of these methods fail because there are only two target points. Based on this, this article proposes a method for horizontal alignment deviation measurement with two target points by using biaxial inclinometer and laser rangefinder data. This method not only meets the measurement accuracy requirements, but also has a fast calculation speed.

VI. CONCLUSION
This study focused on the situation in which there are only two target points and no common target point between the binocular cameras in a binocular position measurement system. The relationship between the two camera coordinate systems, as well as the biaxial inclinometer and laser ranging data are used to determine the horizontal alignment deviation of the moving component. In the Monte Carlo analysis, an accuracy of 2 mm can be achieved at different positions and attitudes or degrees of error (within the allowable range of the system). The experimental results agree with the simulation. In conclusion, the proposed position measurement method with a binocular camera without common target meets the practical application requirements.