A Coarse-to-Fine DLG and TLS Data Registration Method

Terrestrial laser scanning (TLS) is an efficient tool to capture the point cloud for facade modeling. However, it is not easy to obtain the position by Global Navigation Satellite System (GNSS) as occluded by high buildings, which would lead to a problem in orienting the TLS data to a target coordinate system. In this article, we propose a coarse-to-fine registration method for reliably orienting the TLS data using a digital line graph (DLG). The orientation is completed by horizontal plane registration and elevation registration parts. The proposed method is composed of five steps: 1) using the proposed method to extract the corner points from the building outline of DLG; 2) using the proposed method based on RANSAC algorithm to extract the corner points and the facade point clouds of the TLS data; 3) coarse alignment of the two extracted sets of corner points using four-point congruent set algorithm; 4) fine registration by 2D-ICP algorithm; and 5) completing elevation registration using elevation control points. Three datasets are employed to perform the experiments, and the results confirm that the proposed method is reliable and efficient in achieving the registration of DLG and TLS data. After comparing with two other typical registration methods, it is found that the proposed method performs better.


I. INTRODUCTION
T HE development of smart cities and digital twins accelerates the demand for detailed models of buildings. The terrestrial laser scanning (TLS) system quickly and efficiently obtains massive 3-D point cloud data that are an essential supplement for mapping difficult-to-reach locations for the mobile laser scanning system. TLS further provides a data source for the reconstruction of the facade model [1]. The original TLS data coordinates are often defined in the local coordinates. Therefore, they must be transformed into the world coordinate system for subsequent application. Traditional methods often use the Global Navigation Satellite System (GNSS) or total station to measure the targets in the scene for orientating the TLS data. Nevertheless, the accuracy of GNSS measurement in densely populated areas of urban buildings is low as it is affected by nearby tall buildings. Using the total station also needs to start measuring and transmitting from the known control points that are not always available. There exist several approaches for orienting the TLS data. For example, Lu [2] completed the orientation of TLS data using the ALS data. In this method, the orientation of TLS is realized by extracting the line features of the ALS and TLS to perform a semiautomatic registration process. Guo and Cheng [3] also proposed a classification result-guided feature plane registration method to orientate the TLS data by the ALS. Liang et al. [4] proposed a skyline context descriptor to quickly orientate the TLS data to the ALS data. Furthermore, Kato et al. [5] used tree top locations as tie points to orientate the TLS data to the ALS data and UAV-SfM point clouds.
Although these methods have realized the orientation of the TLS data, in many cases, it is difficult to obtain other point cloud data with the absolute coordinate system. Currently, for many urban areas, a large number of 1:500 or 1:1000 digital line graphs (DLGs) are available with abundant and accurate points, lines, and areas information. These data can provide control information for the TLS data.
Several previous research works also tried to use DLG to orient the point cloud. For example, Hu and Dai [6] proposed an automatic LiDAR point cloud registration method based on the DLG. In this method, the Fourier descriptors are adopted to describe the curve features for matching. Also, Tao et al. [7] realized the accuracy evaluation of LiDAR point cloud by comparing the elevation in DLG with the corresponding position in LiDAR point cloud. Using DLG to orientate the TLS data requires performing registration between them. The terrestrial laser scanners often have built-in gyroscopes to ensure the vertically upward direction of the Z coordinate in the TLS data. In such cases, the 3DOF rotations are constrained to only azimuth and the degree of freedom in registration is reduced from 6 to 4 [8]. Therefore, the registration between the TLS and DLG can be carried out in 4 DOF spaces. In other words, this is completed with a 3DOF registration in a horizontal plane and one shift parameter in the Z-direction [9]. The 3DOF registration can be also realized through a 2-D building outline matching process, which can be divided into coarse registration and fine registration.
The building outline matching can be realized by geometric constraint of linear features (such as in [9] and [20]). This method, however, can be easily affected by the line extraction results. Because of the difference between the building outline and DLG, its result is not robust. Extracting the representative feature points from the building outline and DLG to achieve registration will be a good solution.
To address the above problems, in this article, we propose a coarse-to-fine registration method for DLG and TLS data. The main contributions of this article are as follows.
1) We propose a novel coarse-to-fine registration method for TLS data DLG data. The registration process is decomposed into horizontal and elevation registration steps. 2) To carry out the horizontal registration, a building outline matching method based on the building corner points is proposed, which lays the foundation for the registration of DLG and TLS data. 3) A robust elevation registration is proposed to alleviate the possible noise in the TLS data and DLG elevation points. The evaluation of three sets of TLS and corresponding DLG data shows that the proposed method is feasible and efficient. The rest of this article is organized as follows. Section II introduces the proposed method, the experimental data, and the procedure of registration between the DLG and TLS data.
In Section III, we analyze the results. Discussions about the proposed method and the experimental results are presented in Section IV. Finally, Section V concludes this article.

A. Overview of the Approach
The input of the proposed method is DLG data only keeping building polygons and elevation control points and TLS data generated from merging several stations of TLS scanning data. The workflow of this method is shown in Fig. 1. It is composed of the following five steps.
1) Corner points are extracted from the building outline of DLG. 2) Using the proposed method based on RANSAC to extract the corner points and the facade point clouds of the TLS data. 3) Two sets of corner points are initially aligned using the 4PCS algorithm; 4) Fine registration of DLG and facade points cloud is performed using the 2D-ICP algorithm;

B. Data Preprocessing
Along with the revolution of the Lidar scanner, high volume and dense point clouds can be obtained at present, which could be a high burden for subsequent processing. In addition, the resolution varies from the station of the scanner to the position far away from the station. Thus, downsampling processing is employed to reduce the volume of the huge raw point cloud. By considering accuracy and efficiency, a voxel grid filter is used in our pipeline. The whole aligned data are cut into a regular 3-D voxel grid, and only the centroid of the points inside each cell is used to represent the whole voxel. The voxel with size S v (experimentally set to 0.05 m) is used for downsampling. The filtered point cloud will be used as the input for the subsequent processing.

C. Registration Based on Corner Points
Even if voxel filtering is performed, the number of the point cloud remains huge. In addition, the DLG data are represented in vector format, which cannot be directly used for conventional point-based or line-based coarse registration methods. Thus, a corner point-based registration method is proposed to coarsely align the DLG data and point cloud. Especially, the corners of the building which are relatively easier to be extracted from both data. Different approaches are designed for different sources of data. The details are as follows. To illustrate the approach, a pair of test data shown in Fig. 2 is taken as an example.

1) Corner Points Extraction From DLG:
For DLG data, there are several kinds of endpoints. Only the cross intersection will be served as the points for registration, thus other unwanted endpoints have to be removed. The workflow of the extraction process is described in Fig. 3 and the detailed steps are as follows.
1) As for the DLG shape vector, each polyline is expressed as an object. Thus, the polylines are separated and stored in a list LL. 2) Each polyline Li is popped from LL, then the endpoints for Li are separated and stored in a list LP.  3) Each endpoints Pi is popped from LP, then the lines linked to Pi are searched. If only one linked line is obtained, such as point P 4 in Fig. 4(b), the next Pi will be popped and processed. If two linked lines are searched, such as point P 2 in Fig. 4(a), two linked lines P 1 P 2 and P 2 P 3 are obtained for P 2 . Then, the intersection angle A between the two linked lines is calculated and compared to the threshold A c . If |A-180|>A c , Pi is thought to be a corner and pushed into a corner list LC. The process will be continued till the size of the LP is equal to zero. 4) The process will be continued till the size of the LL is equal to zero. 5) Finally, the corners stored in LC are output for the subsequent coarse registration.  2) Facade Point Clouds and Corner Points Extracted From TLS Data: To obtain the corner points of TLS data, we should obtain the facades of the TLS data first. There are several methods to obtain the facades of TLS data, such as the DoPP algorithm [21] and Hough 3-D [22], [23], [24]. The basic idea of DoPP is to divide the TLS area into a grid, then project the point cloud into the grid and calculate the number of points in each grid. The number of points in the grid will be used to determine where the facade is by comparing the number with a given threshold. It can extract the precise facade points cloud by increasing the threshold, but its facade results are incomplete and discontinuous which leads to poor performance. For the Hough transform-based method, the results are sensitive to the segmentation parameters values [24]. When using those results to calculate the corner points and participate in registration, the accuracy can be easily affected. To address this issue, we propose a RANSAC-based method to extract corner points and the facade points cloud. The workflow is illustrated in Fig. 6. The details are as follows.
1) To get the precise facade results, the ground points are separated by employing the CSF algorithm [25], then the remaining unground points are used for subsequent processing. Facade points are extracted from the unground points by a RANSAC approach. For robust plane detection, the method proposed in [26] is used (to obtain the exact plane and accurate facade point clouds, we set the max distance to fitting(D R ) to 0.1 m). After performing the RANSAC shape detection processing, a minimum bounding rectangle is fitted to each plane. For each rectangle, its normal is calculated and compared to the z-axis. If the angle exceeds a given threshold A f (experimental set to 70°in this article), we consider this rectangular plane as a facade and save the points participating in the rectangular plane fitting. Fig. 7 shows the facade extraction results of Fig. 2(a). After removing the ground points by the CSF algorithm, plane detection is performed on the remained point clouds; there are 11 planes detected as shown in Fig. 7(a). From the results, it can be found that some planes do not belong to the facade. After removing the unwanted planes, nine facades were extracted as shown in Fig. 7(b). Fig. 7(c) shows the top view of the facades extracted results; it can be found that the results can reflect the building outline clearly, which provides a convincing basis for the subsequent building corner detection.
2) The four vertices of each fitted rectangle are projected onto the horizontal plane, and the midpoint of two adjacent points (the two points on the vertical edge will be very close or even coincide) is calculated and the two midpoints are linked to get the facade line segment. 3) Calculate the intersections and select the corner points, the details are as follows: Step 1. Calculating the intersection of extension lines of line segments.
Step 2. Determining the positional relationship between the intersection and the line segments through the vector angle between the intersection and the endpoints of the line segment. Take Fig. 8 as an example, the angle between vector − → oa and vector − → ob is 0, and the angle between vector − → oc and vector − → od is π, so the intersection o is on the line segment ab and outside the line segment cd.
Step 3. If the intersection is on two line segments, consider it a corner point [see Fig. 9(a)]. If the intersection point is on one of the line segments, calculate the distance between the intersection and the adjacent endpoint of another line segment, and if the distance is less than the threshold D p (D p is set to 1.0 m), consider it as a corner point [see Fig. 9(b)]. If the intersection is not on two line segments, calculate the distance between the intersection and the adjacent endpoints of two line segments. If those two distance values are less than the threshold D p , the intersection is considered as a corner point [see Fig. 9(c)]. Fig. 7(d) is the facades line segments of Fig. 7(b); after selecting the corners, there are eight corners.
3) Coarse Registration Based on 4PCS: 4PCS is a fast, robust, and antinoise alignment scheme for 3-D point sets. To use the source of the 4PCS algorithm, zero is added to the corner points from DLG data and TLS data to form the pseudo 3-D points.
The algorithm selects three points (a, b, c) from the points cloud randomly to form a plane, with the requirement that the sides of the triangle consisting of point a, point b, and point c should be as long as possible. Then, it finds the four (approximately) coplanar point d to constitute a base B = {a, b, c, d} (see Fig. 10). In another point cloud, find the set of bases that (approximately) equal to B, calculate the rigid body transformation matrices and get the best transformation matrix.
The corner points we used are all on the same plane (Z values are all 0), so the procedure of finding coplanar point is omitted. Furthermore, because the corner points are representative and the number of points is small, the coarse registration process will be very fast.

4) Fine Registration Based on 2D-ICP:
After coarse registration via the 4PCS algorithm based on extracted building corners, the next step is to precisely align the TLS data to DLG. Based on the coarse registration, TLS data are nearly aligned to DLG data, thus we use the ICP algorithm to refine registration as a good initial parameter from form steps. The fine registration is also performed in 2-D. To complete the 2D-ICP. The DLG data and extracted building outlines from the TLS data are both discretized into points at a distance of 0.1 m.
The relationship between the two points is expressed by where X and Y are 2-D coordinates of the discrete points of DLG data, X and Y are 2-D coordinates of the discrete points of building outline from the TLS data, θ is the rotation angle, and T X and T Y are the shift parameters.
To improve the speed and accuracy of the 2D-ICP algorithm, the KD tree is used to search for the closest points. The closest point search distance is gradient decreased after each iteration. Because after each iteration, the source point cloud is closer to the target point cloud, and the closest point with a shorter distance has higher reliability (the initial distance (D i ) is experimentally set to 10 m, and descending D d (set to 0.5 m in this article) per iteration until up to D m (set to 1 m in this article).

5) Robust Elevation Registration:
After performing 2D-ICP on the point cloud, the translation parameter along the elevation direction is calculated by directly searching the TLS data and the elevation points in the DLG data.
As shown in Fig. 11, for the elevation point P (X P , Y P , Z P ), the closest point in the plane distance (D z ) is selected as point    A. The elevation offset ΔZ P of point A is then calculated as ΔZ P = Z P −Z A . After calculating all the offset values for all the elevation points, the average value of ΔZ P is calculated by considering the gross error. The gross errors are then deleted by comparison based on the maximum-minimum value with the mean value of all the offset values. If the difference between maximum-minimum value and mean value exceeds a given threshold T z , this maximum-minimum value is considered to be a gross error. These gross errors are then deleted. This process is repeated until all the gross errors are removed (see Algorithm 1). We then use ΔZ P as the global elevation offset parameters to realize the elevation registration.

A. Experiment Data
To evaluate the proposed method, three sets of TLS data are used for the experiments. They are Area-A [see Fig. 12(a)], Area-B [see Fig. 12 Table I. The corresponding DLG from some historical database is shown in Fig. 13. The DLG accuracy of Area-A is 0.1 m, and the DLG accuracy of Area-B and Area-C is 0.05 m. There are some elevation points in the DLG as shown as black dot points.

B. Evaluation Criteria
To evaluate the plane registration accuracy, we use manual point selection registration and 2D-ICP fine registration to register the TLS data and DLG to obtain the reference matrix. Then, we compare the matrix calculated by the proposed method with the reference matrix to get the rotation difference and displacement value.
In the vertical direction, the elevation registration accuracy is validated by leaving one cross-validation.

C. Parameters Setting
It is worth mentioning that the method is not sensitive to the parameters, the reference values of the key parameters in the experiment are summarized in Table II of each process.

D. Facade Point Clouds and Corner Points Extraction Result
After obtaining the facade rectangle, the facade point clouds (see Fig. 14) and facade line segments (see Fig. 15) are obtained. Based on the facade lines, the corner points are obtained and  illustrated as a blue solid circle. Because d m is small and some walls are uneven, so there is a phenomenon that multiple line segments are on one wall as shown in Fig. 15(b). However, this will not affect the coarse registration. Fig. 16 shows the corner points extraction from DLG. In Fig. 16, the blue solid circle represents all endpoints in DLG and the red triangle represents the extracted and retained corner points. It can be seen that all corner points are reserved, and redundant endpoints [particularly the endpoints on the dotted line in Fig. 16(b)] are removed by the proposed method. By manual checking, all the retained endpoints are correct building corner points. Fig. 16(a) shows the corner points extraction of Area-A, there are 93 endpoints in the DLG data (shown as blue dot points), and after removing some unwanted points, there are 87 corner points (shown as a triangle) retained. For Area-B, there are 152 endpoints, after processing, 41 corner points are obtained in Fig. 16(b). By visually checking, all the retained corner points are real building corners. For Area-C, there are 49 endpoints as shown as a blue circle in Fig. 16(c); from the result, we can see that there are mainly endpoints with only one line linked. After processing, these endpoints are removed, and only 30 corners are retained. The retained corners are kept for the subsequent coarse registration based on 4PCS.

E. Registration Result 1) Coarse Registration Result:
After extracting corners from both data, the coarse registration is performed, and the results are shown in Fig. 17. In Area-A, its facades and DLG aligns well. In Area-B and Area-C, the facades are partially missing and redundant. It can be seen that the proposed method can still provide good initial registration results for subsequent processing. It is worth mentioning that, if the initial registration parameters are obtained by other means, the fine registration in the next step can be performed directly by skipping this part.
2) Fine Registration: Fig. 18 illustrates the results of fine registration based on 2D-ICP of three datasets. In these figures, the DLG is shown in red color and the TLS point cloud is   illustrated in blue color. As shown in Fig. 18, three sets of TLS data and DLG are well registered together.
In Figs. 19 -21, the details before and after fine registration are compared. To examine the details, two typical areas are marked as block 1 and block 2 in Fig. 19(a). Fig. 19(b) and (c) shows the magnified view of the subarea in Fig. 19(a). From the magnified view, we can find that the line in red color (belonging to the DLG data) is nearly close to the point cloud. Fig. 19(e) and (f) shows the magnified view of the subareas in Fig. 19(d). The results show that DLG data are attached to the building outline of TLS To evaluate the accuracy, we use manual point selection registration and 2D-ICP fine registration to register the TLS data and DLG to obtain the reference registration parameters. The detailed parameters for the 2D-ICP registration are shown in Table III. The search distance is decreased from 10 to 1 m. For the three datasets, the rotation difference is −0.095°, −0.033°, and 0.013°, for which the displacement is 0.054 m, 0.072 m, and 0.050 m, respectively. Fig. 22 shows the result of elevation registration based on the elevation points in the three areas, where the elevation points and TLS data are marked in red and green color, respectively. T z is experimentally set to 0.2 m. From the side view of the registration result, it can be found that the elevation points are well-aligned with the TLS data. The elevation registration was validated by leave-one-out cross-validation, and the accuracy of the Area-A, Area-B, and Area-C is 0.102 m, 0.024 m, and 0.126 m, respectively.

3) Elevation Registration:
The above results confirm that this elevation registration method is highly accurate.

F. Comparison With Other Methods
For comparison, we also implement coarse registration directly using all the facade points cloud and DLG. After many times of parameters adjustment, it also could obtain similar results as shown in Fig. 17 and lead to a convergence of fine registration based on ICP. But it took several minutes to tens of minutes. Its results are not as stable as the results of the proposed method based on corners. As the efficiency of plane detection based on the RANSAC approach and the number of points is greatly reduced after corner points extraction, the process of corner points extraction and coarse registration altogether only took a few seconds as shown in Table IV.  Furthermore, graph matching [28] with corner points was used instead of 4PCS for coarse registration, and the results showed that only the result of Area-A can be used as the initial value of subsequent 2D-ICP registration. The accuracy comparison of 4PCS and graph matching is shown in Table V.
In addition, we project the point clouds on the horizontal plane and then search for line segments by Hough transform [29] to compare with the RANSAC-based method in this article. The results show that the Hough 2-D method can detect some facade line segments (see Fig. 23), and 4PCS algorithm can also provide useful initial values (see Fig. 24) using the corner points calculated by its extracted line segments. Under the data in this article, its efficiency is the same as that of the method proposed in this article. However, compared with the face detection method based on RANSAC in 3-D space in this article, the Hough 2-D method lacks 1-D information, resulting in some missing and false detection, and its line segments extraction effect (see Fig. 23) is not as good as that by the RANSAC-based method in this article (see Fig. 15).
The above comparisons confirm that the 4PCS registration based on facade point clouds is feasible but not efficient. We could also perceive that the corner points extraction is an essential and valuable step in this process. The 2-D registration method based on corner points can provide highly efficient and stable results.

IV. DISCUSSION
In the orientation task of TLS point cloud, because DLG is easy to obtain, the data are small but the geographic information  is accurate; it is an economical and feasible choice to use DLG as a reference to orient TLS accurately. In this article, we propose a coarse-to-fine registration method of TLS point cloud and DLG, which makes full use of the characteristics of two data and reduces the standard registration of 6 DOF to 4 DOF.
During the experiment, we found that even if the TLS data have some defects (due to occlusion or insufficient stationing), or the building structure in the TLS data is much more complex than the DLG, the coarse registration can be completed by our proposed method, providing a good initial value for 2D-ICP.
The wall in DLG is linear, but in fact, the wall is a certain thickness and uneven; so in Fig. 21, it can be seen that after the fine registration, TLS data and DLG are not perfectly superimposed together. However, the 2D-ICP is a globally optimal solution under a good initial value, and its search range is gradually decreasing, which can ensure the optimal registration of TLS data and DLG.
Based on the above discussion, the elevation registration in the vertical direction is carried out. For each control point, the search space is a cylinder (see Fig. 11), ideally, the closest point found can be used as a correct reference. However, due to the presence of noise, we designed Algorithm 1 to avoid the influence of noise and perform robust elevation registration.

V. CONCLUSION
In this article, we proposed a coarse-to-fine registration method for DLG and TLS data. In our approach, the registration is decomposed into a 3-DOF 2-D building outline matching and one-shift parameters estimation in the Z-direction. A cornerpoint-based matching method was proposed for efficient and reliable matching of the building outline. The performance of the proposed method was evaluated by three sets of TLS and DLG data. The results confirm the efficiency of the proposed method in obtaining highly accurate registration results.