Image Precise Matching With Illumination Robust in Vehicle Visual Navigation

In vehicle visual navigation, image matching algorithm is highly critical to positioning accuracy and processing efﬁciency. One single matching algorithm cannot satisfy all types of image features accurate acquisition, so Harris, SUSAN, FAST, SIFT


I. INTRODUCTION
Image matching is one of key technologies in the study of vehicle visual navigation, which its result directly affects the accuracy of vehicle positioning and navigation. Because the images collected in vehicle actual driving are often interfered by illumination change, the image precise matching algorithm with robust illumination is very important to vehicle visual navigation. At present, the classical algorithms are mainly as follows: Lowe proposed the SIFT (Scale Invariant Feature Transform) algorithm [1], which integrated points detecting, feature vector generating and feature matching for optimization and achieved a good matching performance. Ke et al. proposed the PCA (Principal Component Analysis)-SIFT algorithm [2], which improved the matching speed of feature points by extracting feature vector through the The associate editor coordinating the review of this manuscript and approving it for publication was Zhihan Lv . patch near the pixel, which reduced the descriptor dimension. Bay et al. put forward the SURF (Speeded Up Robust Features) algorithm [3], which approximately simplified the Gaussian second-order differential template in DoH and further improved the running speed drawing on the idea of approximate simplification in SIFT. Mikolajczyk et al. proposed the GLOH (Gradient Location Orientation Histogram) algorithm [4], which extended the SIFT feature descriptor, enhancing robustness and uniqueness. Yang et al. proposed a new local invariant feature detection and description algorithm [5], which the descriptor was generated based on the distance and direction histograms of the gradient. It had a lower feature vector dimension, which was helpful to improve the speed of image feature matching. The performances of various matching descriptors were compared and analyzed in the literatures [6], [7], including image rotation, compression, perspective change, scale change, blur change, and illumination change. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ There is the interference of illumination change when collecting images in vehicle positioning, which is not conducive to image feature matching. The typical matching algorithms such as SIFT and SURF suffer from accuracy reduction significantly in illumination large change. To solve the problem of image matching in illumination change condition, Xiao et al. put forward an ambient light independent image mosaic algorithm [8]. Its feature points were extracted by ring projection, which overpassed the local area limitation in tradition methods and gained more accurate image matching for image under illumination change. Wu et al. proposed a color SIFT matching algorithm under complex illumination changes [9], in which the geometric invariance was obtained on the images' color invariant information to accomplish matching. It could get more reliable matching data to elevate the rate of correct image recognition. Ling et al. [10] proposed a robust multi-level scene matching algorithm combining infrared and visible light, which introduced phase consistency transformation of illumination and contrast invariance into multi-level scene matching. The matching accuracy was high, but it was difficult to choose the appropriate filtering wavelength. Zhao and Wang [11] put forward an improved KAZE algorithm to match images with illumination change. The descriptor is constructed by a second-order feature with low sensitivity to illumination change. Its accuracy had been improved in some scenarios like over exposure, under exposure, and non-uniform illumination weakening, and it had better robustness. Liu et al. [12] proposed an image matching method of new illumination-robust and anti-blur combined moment invariants, which constructed the combined invariants with illumination robustness and fuzzy invariance based on Hu invariant moments. Its matching speed was greatly improved under the premise of high matching accuracy. Lei [13] proposed a stereo matching algorithm based on occlusion effects and illumination change. By calculating the cumulative distribution function of the stereo image histogram under different illumination conditions, the adverse effects of different illumination conditions on stereo matching were reduced and its robustness was enhanced. Zhang [14] reduced the number of sub-pixel regions by redividing the rectangular pixel regions in SIFT algorithm and merged vector correlation coefficients at the feature matching stage to filter the matching points that could not be judged by Euclidean distance. Because the constraint condition of the vector correlation factor was enlarged, it significantly reduced the total number of redundant feature points, and largely eliminated the problem of mismatched points caused by region similarity, which improved the correct matching rate. Wang et al. [15] extracted the feature points with illumination invariance by simulating the effect of illumination change on the image, and chose local intensity sequence as the descriptor. It yielded a great progress in image matching accuracy, but the matching speed was not comparable enough. Ma et al. [16] put forward an road edge detection algorithm with illumination robust, which was used to solve the adverse effect of illumination changes and heavy shadow. Zhang et al. [17] tested the combination of several image enhancement technologies and SURF to extract image feature, and then used RANSAC to filter. The results showed that image enhancement was beneficial to feature extraction. Therefore, it is of great practical significance to explore a feature matching algorithm with high matching efficiency and invariance of illumination change.
This paper focuses on image matching algorithms in vehicle visual navigation. For the different modes of images collected under normal illumination, five classical feature point matching algorithms are selected for qualitative and quantitative analysis. To solve illumination changes in the collected images, an image precise matching algorithm with illumination robust is proposed. The image is enhanced by preprocessing, and the SURF feature points are optimized based on the idea that the gradient information is less sensitive to illumination change, then the bidirectional search is used to achieve accurate matching. The matching speed is greatly improved while ensuring a high matching accuracy.
The remainder of this paper is organized as follows: The general idea of image matching algorithm under normal illumination is briefly introduced in Section II. The detailed process of the image matching algorithm with illumination robustness is presented in Section III. The experimental setup and extensive examples of the system performance are provided in Section IV. Finally, the conclusion is drawn in the last section.

II. IMAGE MATCHING ALGORITHM UNDER NORMAL ILLUMINATION
The images collected under normal illumination conditions in vehicle driving are continuously changing. Four kinds of pavements (Asphalt, Cement, Square brick, and Hexagonal brick) are considered as analysis objects, as shown in Fig.1. Due to the diverse image type in vehicle positioning technology, the representative algorithms as Harris, SUSAN, FAST, SIFT, and SURF are chosen to process the images in order to find effective application, and the acquired feature point information is qualitatively and quantitatively analyzed. The algorithm used in actual application is selected by its detection rate and operating time.

III. IMAGE MATCHING ALGORITHM WITH ILLUMINATION ROBUST A. IMAGE PREPROCESSING
For images with illumination change, histogram equalization [18] is one of the important waysfor image preprocessing. Its basic idea is to transforman original imagewhose gray probability distribution is randomized to a new image whose gray probability distribution is balanced through some gray transformation, as shown in Fig.2. After histogram equalizationprocessing, the image pixels will occupy most of the gray levels andbe distributed evenly. Thus, the enhanced image has highercontrast and its gray leveldistribution is more uniform.  In Fig.3, (a) is the original image. (b) is the preprocessed result, whose contrast is enhanced obviously and the gray leveldistribution is more uniform. However, there is large spacing between its informationin low frequency and high frequency. In order to achieve more attractive balance, a fusion method with dynamic weight is applied by retaining some gray-level informationfromthe original image, which is defined by equation (1). Fig. 3(c) indicates the fusion result.
where n 1 is the mean gray value of the original image I ;n 2 is the mean gray value of the image h_I processed by histogramequalization; end_I is the result after preprocessing. When n 1 /n 2 is closed to 1, n 2 − n 1 /n 2 approaches 0, It means that the closer the average gray value of the original image is to the enhanced image, the greater the proportion it will take in the fusion process.

B. IMPROVED IMAGE FEATURE MATCHING ALGORITHM
The SURF algorithm is translation, rotation, and scale invariant, and has high processing efficiency, but it is not robust to illumination change. It's known that the image edges and details are less sensitive to illumination change, so in order to overcome its effect, the SURF feature points are optimized by image gradients based on the idea of Canny operator [19]. Bidirectional search is also used to achieve precise matching, whichguarantees a high matching speed and meanwhile improves the matching accuracy.

1) IMPROVED SURF FEATURE POINT DETECTION
(1) Use SURF algorithm to detect candidate feature points for the preprocessed image.
where σ is the smoothing parameter.
(3) Assume pixel (x, y) as a candidate feature point, the amplitude and direction of its gradient is calculated by VOLUME 8, 2020 equation (3) and (4).
where G y (x, y) and G x (x, y) are the partial derivatives of vertical and horizontal directions, they are defined as: (4) Judge whether the candidate feature point is a local maximum along the gradient direction, as shown in Fig.4. Specifically, compare the gradient value at the current position M (x, y) with the gradients at the intersection of the two sides in the gradient direction flag 1 and flag 2 . Because the intersection point is not in the 8-neighborhood of the current pixel, a linear interpolation is performed on the gradient magnitude of the pixel at both ends. If M (x, y) > flag 1 and M (x, y) > flag 2 , the pixel will be kept as a feature point; otherwise it will be discarded.

2) MATCHING ALGORITHM OPTIMIZATION
The similarity of two different feature points is generally calculated by Euclidean distance, if the ratio of the minimum Euclidean distance to the second smallest one is less than a certain threshold T 1 (T 1 = 0.6), this pair of matching points is accepted. By using bidirectional search, the correct matching accuracy can be Significantly increased. The specific steps are as follows: (1) Calculate the distance of all feature points between two images, which is given by equation (7).
where, the feature point number of the first image is m, and the second is n, d ij (i = 1, 2, · · · , m; j = 1, 2, · · · , n) represents the Euclidean distance between the feature point i in the first image and the feature point j in the second.
(2) Obtain valid matching points through bidirectional search.
For each feature point in the first image, D is sorted in descending order by row to search its corresponding matching point in the second image, then the D_row is obtained. If d_row i2 /d_row i1 < T 1 (i = 1, 2, · · · , m) is fulfilled, it can be thought that there is a matching point in the second image corresponding to the feature point of the first image, and put thispair of matching points into the set M 1 which distance is d_row i1 . Conversely, the matching point of feature point i does not exist.
For each feature point in the second image, D is sorted in descending order by column to search its corresponding matching point in the first image, then the D_col is obtained. If d_col 2j /d_col 1j < T 1 (j = 1, 2, · · · , n) is fulfilled, it can be thought that there is a matching point in the first image corresponding to the feature point of the second image, and put thispair of matching points into the set M 2 which distance is d_col 1j . Conversely, the matching point of feature point j does not exist.
(3) Obtain the set of matching points The sets M 1 and M 2 are obtained by the bidirectional search above, and their intersection is calculated to get the final matching points, which ensuresits accuracy, as defined by equation (8). Points in both M 1 and M 2 are the matching feature points, and they are used to ensure the correctness of the matching effect. In order to visually describe the relationship between the number of feature points and the running time, the efficiency function P is defined as shown in equation (9), where N represents the number of feature points, and T is the corresponding processing time (unit: s). The larger the value of P, the better the performance of balancing the number of feature points and running time. The above five algorithms are used to perform feature point detection on the first image of each group in Fig.1, and the results are shown from Fig.5 to Fig.9. The performances of their several parameters are compared in Table 1.

IV. EXPERIMENTAL RESULTS AND ANALYSIS
It can be seen from Table 1 that for asphalt and square brick pavement images, FAST algorithm detects the largest number of feature points, followed by SIFT algorithm. Harris, SURF, SUSAN and other algorithms have the same magnitude of detected feature points, but SURF has the least running time, and then there is FAST, Harris, SIFT, SUSAN in proper order. For cement pavement, Harris is the best for detected number,       brick pavements have the maximum number of feature points; SURF has the shortest time to detect feature points in any type pavement. To achieve comprehensive performance, SURF is optimal and FAST is the second.
A few pavement images are not representative enough, so the paper tested 320 asphalt pavement images, 450 cement pavement images, 270 square brick pavement images, and 430 hexagonal brick pavement images. The results of different feature detection algorithms on the same pavement are shown below. Fig.10 -Fig.12 compared the number of feature points,the running time, and the efficiency P of these algorithms on the four pavements respectively. Fig.10 -Fig.11 show that for asphalt pavement, FAST detects the largest number of feature points, SIFT gets the second; SURF is optimal in time efficiency, followed by Harris and FAST. For cement pavement, Harris detects the most feature points, the next is FAST; SURF has the least running time, FAST and SIFT are the second. Tests on square brick pavement show that FAST gets feature points slightly more than SIFT, SURF uses less time while FAST and Harris uses higher. For the hexagonal brick pavement, Harris detects the most feature points, and the other methods have less than 120 feature points. There are SURF, SIFT, and FAST in ascending order for running time. It is easy to observe from Fig.12 that SURF has the largest efficiency P value overall for the variety of pavements, and its comprehensive performance is the best; the next is FAST. In short, based on the performances such as the number of feature points, running time, and efficiency, an optimal feature point detection algorithm can be selected in actual vehicle positioning process according to the practical situation to complete the image matching.
Among above algorithms, Harris, SUSAN, and FAST are feature point detection algorithms, so it is necessary to further add appropriate feature point descriptors and similarity measuring algorithms to achieve image matching when are applied to vehicle positioning. SIFT and SURF are two image matching algorithms after integration and optimization. The four groups of images in Fig.1 with a certain coincidence are tested for matching performance, the ratio Pre of the number of mismatched points and the total is as a standard, as shown in equation (10), where false_matches represents the number of incorrect matches. There are too many matching points between images for human eyes, so this paper uses the original RANSAC algorithm with constant parameters to judge whether it is a correct match. Their results are shown in Fig.13, and the performance parameters are shown in Table 2.
In Table 2, Total indicates the total number of matching points, Num refers to the correct matching points, Time means the running time, and Pre is the ratio. The analysis shows that the SIFT has a large number of feature point bases, so it achieves a large number of matching points and higher accuracy rate, but correspondingly has large time consumption; the SURF has relatively fewer matching points and lower matching rate, but it has high processing     efficiency that meets the needs of vehicle positioning. Besides, it is also more applicable to the hexagonal brick pavement.

B. PERFORMANCE ANALYSIS OF IMAGE MATCHING ALGORITHM UNDER ILLUMINATION CHANGE
The experiment in this section is completed in three parts. In the first part, we verify the performance of the proposed improved SURF algorithm on detecting feature points, and compare it with FAST, SIFT, and SURF. In the second part, the matching accuracy of bidirectional search is verified. In the third part, the improved SURF algorithm is used to detect feature points, the points are matched based on bidirectional search, the feature matching is performed on various images affected by different levels of illumination change, and compares with SIFT and SURF. Taking into account the actual situation in vehicle positioning process, two kinds of images are tested. Fig.14 is the asphalt  [20]. The image that meet the condition g > T 2 , T 2 = 0.25(See equation (11)) is considered to be disturbed by illumination changing and affect the positioning result, and is obtained through a large number of experiments, so the image matching algorithm with illumination robust is adopted, and on the contrary, the illumination influence is not considered.

1) PERFORMANCE VERIFICATION EXPERIMENT OF FEATURE POINT DETECTION
To verify the performance of the feature point detection algorithm, FAST, SIFT, SURF and other algorithms are applied to detect feature points in Fig.14 -Fig.15, which results are shown in Table 3. If the P value is larger, the algorithm performance of balancing the number of feature points and running time is the better. Through analyzing Table 3, the number of feature points detected by FAST, SIFT, and SURF algorithms decreases significantly as the images get darker, while the SURF with the proposed image preprocessing and the study of this paper show better robustness to illumination changes and maintain relatively small fluctuation of feature points number.  The algorithm in this paper is an optimization based on SURF, which increases the amount of data processing but maintains a little change in running time. Although the number of feature points is reduced, the reserved feature points are more robust to illumination, compared with the SURF (+ preprocessing). Fig.15(a) is the template image, Fig.15(b)-(f) are image to be matched. Based on the proposed feature points and SURF descriptor, Euclidean distance is used to measure similarity. Selecting the original RANSAC algorithm with constant parameters as the unified standard for calculating the mismatch points, the normal unidirectional search and bidirectional search are used to match feature points respectively, and the results are shown in Table 4. It is easy to know that the matching accuracy is improved greatly in bidirectional search, although the number of matching points is reduced. Conclusively, the bidirectional search effectively reduced the use of matching optimization algorithms. VOLUME 8, 2020

C. VERIFICATION EXPERIMENT OF IMAGE FEATURE MATCHING PERFORMANCE
In order to verify the matching performance, SIFT, SURF and the study of this paper are used in Fig.14 and Fig.15, and the results are shown in Table 5.
According to the analysis, SIFT is based on the idea of gradient and has invariance of illumination, but its efficiency is worse than SURF. At the same time, the image is darker, the gradient and gray information will be seriously lost, making it difficult to obtain the gradient and decreasing the matching accuracy sharply. The paper significantly improves the matching effect on image with illumination change. First, the image is equalized and the fusion progress with dynamic threshold is subsequently used to increase the overall brightness of the image. Then, the feature points are detected by SURF algorithm optimized by gradient information. Finally, bidirectional search is adopted to match. Experiments show that the algorithm can well overcome the problem that the image cannot be matched due to the influence of fuzzy noise and underexposure. It ensures a high matching speed, and greatly improves the matching accuracy.

V. CONCLUSION
In vehicle visual navigation, one single matching algorithm cannot satisfy all types of image features accurate acquisition. Harris, SUSAN, FAST, SIFT, and SURF are used to process a variety of road images under normal illumination condition. When practical application, the optimal algorithm can be selected by the matching accuracy and running time. In addition, because the collected images are often interfered by illumination change in vehicle actual driving, the traditional algorithms do not work well, an image matching algorithm with illumination robust is proposed. Firstly, the image is enhanced by preprocessing, and then SURF feature points are detected and optimized by image gradient based on the idea of Canny operator. Finally, the bidirectional search is applied to achieve precise image matching. The experiments show that the feature points detection and matching remain good stability when the image brightness changes, overcoming the adverse effects of illumination change. Compared with the SURF feature matching method, the research is more robust for illumination change, and has a higher correct matching rate and matching efficiency. Although the accuracy is similar to SIFT, the running time is greatly improved.
CHENG XIN received the B.S. degree in software engineering from Chang'an University, Xi'an, China, in 2012, and the M.S. and Ph.D. degrees in traffic information engineering and control from Chang'an University, in 2014 and 2017, respectively. In 2017, he joined Chang'an University, where he has been an Assistant Professor, since 2019. His current interests include artificial intelligence and vehicle networking technology, high-precision positioning and environment perception for connected and automated vehicles, traffic big data analysis, and information fusion.
HAN RUIZHI received the B.S. degree in computer science and technology from Chang'an University, Xi'an, China, in 2017, where he is currently pursuing the master's degree in traffic information engineering and control. His main research interests include deep learning and computer vision.
ZHAO XIANGMO (Member, IEEE) received the B.S. degree from Chongqing University, China, in 1987, and the M.S. and Ph.D. degrees from Chang'an University, China, in 2002 and 2005, respectively. He is currently a Professor and the Vice President with Chang'an University. He has authored or coauthored over 130 publications and received many technical awards for his contribution to the research and development of intelligent transportation systems. His research interests include intelligent transportation systems, distributed computer networks, wireless communications, and signal processing.