A Feature Points Extraction Algorithm Based on Adaptive Information Entropy

Feature points loss and images mismatch in the variation of light intensity, weak texture and large angle rotation for the feature points extraction of ORB-SLAM2 are severe. To deal with the problem, a feature points extraction algorithm based on adaptive information entropy, i.e., Adaptive Information Entropy Feature (AIEF) algorithm is proposed. According to the information entropy, the image blocks with less information are removed and those with more texture image information and larger gradient are selected. Then an adaptive algorithm is used to automatically calculate the optimal threshold of the image information entropy. The image blocks are homogenized to avoid that the extracted feature points are too dense and getting stuck is prevented, which makes the algorithm more robust. Finaly validation is performed using the Oxford standard data set and the performances of the AIEF algorithm are compared with those of the SIFT, SURF, and ORB-SLAM2 algorithms. Experimental results on the Oxford standard data set demonstrate that the AIEF algorithm outperforms the traditional counterparts in terms of processing time, number of feature points, correct matching number and correct matching rate.


I. INTRODUCTION
Image matching is the process of identifying points with the eponymous end between two or more images through a certain matching algorithm [1], [2]. It is based on the correspondence, similarity and consistency of the image content, features, structure, relationship, texture and gray level. Image feature points detection is a key issue in Simultaneous Location and Mapping (SLAM) [3]. It is extremely important for pose estimation, map creation, and loop detection. A typical scenario of SLAM is [4]: when a mobile robot enters an unknown environment, uses laser or visual sensors to determine its pose and position, and reconstructs a three-dimensional map of the surrounding environment in real time. At present, the technology of SLAM based on laser sensors [5] is quite mature but the cost of it is high. While the cost of vision sensors is low and can obtain richer The associate editor coordinating the review of this manuscript and approving it for publication was Donato Impedovo .
information. The progressive development of visual SLAM makes the method of images matching by feature points detection become increasingly significant [6], [7]. Harris et al. proposed the Harris corner and edge detector algorithm, which mainly uses differential operation and auto-correlation matrix for corner detection [8]. Shi-tomasi corner detection algorithm proposed by Jianbo Shi and Tomasi is an improvement on the Harris algorithm [9]. In [10], David Lowe proposed the Scale Invariant Feature Transform (SIFT) algorithm, which is the most stable algorithm for point detection. It was applied in the first real-time monocular Visual SLAM (VSLAM) [11] system proposed by Davison et al. in 2007. Herbert Bay, on the basis of the SIFT algorithm, improved it and realized Speed Up Robust Features (SURF) algoritm [12] which runs three times faster than SIFT; Murray and Klein proposed the Parallel Tracking And Mapping (PTAM) system. Its innovation is to divide tracking and mapping into two parts and it is based on the FAST (Features from Accelerated Segment Test) feature points detection [13]. After years of development, researches of SLAM have also achieved many new results in recent years. For example, Raul Mur-Artal et al. proposed the ORB-SLAM2 [14] system based on ORB [15] (Oriented FAST and Rotated BRIEF) which is divided into two parts: Oriented FAST feature point extraction and Rotated BRIEF feature point description. ORB has better performance in real-time and speed of extraction than previous works such as SIFT, SURF. The Aerial Robotics Group of the The Hong Kong University of Science and Technology (HKUST) proposed a robust and versatile Monocular Visual-Inertial State (VINS-Mono) estimator [16] -a monocular visual inertial navigation system based on the real-time SLAM framework.
The ORB-SLAM2 can be applied to monocular, binocular and RGB-D cameras. It is a complete visual SLAM system. It has better performance and higher accuracy than previous works, and it can be carried out by a standard CPU in real time. However, there are still some problems to be solved in the ORB-SLAM2 algorithm, e.g., robustness of feature points extraction under sudden changes in lighting or weak textures. In dynamic environments, such as large rotation of camera makes the problem of missing feature point, and the impact on feature point extraction with dynamic object movement in the field of view more severe. In order to solve the problems above, it is a feasible attempt to modify the feature point extraction algorithm of ORB-SLAM2 [17], [18]. We propose the Adaptive Information Entropy Feature (AIEF) algorithm which combines the information entropy [19], [20] and adaptive methods [21] with the feature point extraction algorithm of ORB-SLAM2.
The major contributions of this paper are summarized as follows: a. In this paper, we study and analyze the problem of losing tracking in ORB-SLAM2 algorithm when the camera rotates at a large angle. In order to improve the performance of feature point extraction in ORB-SLAM2, a new feature point extraction algorithm combining adaptive information entropy AIEF, is proposed.
b. Considering the image mismatch in terms of image blur, illumination change, image rotation, affine transformation, etc., we use the proportional-based feature point homogenization algorithm instead of the quadtree homogenization algorithm in ORB-SLAM2.
c. To avoid setting the threshold of entropy artificially with empirical value, an adaptive information entropy algorithm is adopted to obtain the best entropy threshold of each image block automatically.
Based on the above improvements, we can obtain more good quality feature points. And the feature point matching rate of the AIEF algorithm is improved compared with the SIFT, SURF, ORB-SLAM2. Experimental results show that the AIEF is 2.75%, 13.12% and 25.48% higher than ORB-SLAM2, SIFT and SURF algorithm respectively on average.

II. SYSTEM MODEL AND RELATED PRINCIPLE
The ORB feature points extraction is composed of oriented FAST corner extraction and rotated Binary Robust Independent Elementary Features (BRIEF) [22] descriptor calculation. It can calculate quite fast and has rotation invariance, but no scale invariance. Also, It can run in real time in a narrow indoor environment and a wide outdoor environment.
The diagram of system model is shown in Fig.1. Firstly, the image pyramid is built by ORB-SLAM2 feature point detection algorithm. Secondly, FAST feature points in pyramid grid are extracted. The feature points extracted by FAST algorithm are homogenized by quadtree homogenization algorithm [23]. The rotation angle of the feature points is calculated by the center of gravity method, and the feature points are described by the BRIEF algorithm. Finally, feature points matching and image matching are completed.

A. CONSTRUCT THE IMAGE PYRAMID
In order to make feature points scale invariant, ORB-SLAM2 constructs an image pyramid. The bottom layer of the image pyramid is the input image, and the n-th layer image is obtained by multiplying the (n-1)-th layer image by the scaling factor s. Set the length of the input image to L and the width of the input image is expressed as W, thus the length and width of the n-th layer images are expressed as: Therefore, the image pyramid, i.e., the area of the n-th layer is Obviously, the higher the number of image layers, the smaller the image area of this layer. Therefore, the fewer image blocks are segmented and the fewer feature points are extracted. VOLUME 8, 2020

B. HOMOGENIZATION ALGORITHM
The ORB-SLAM2 used the quadtree homogenization algorithm to make the distribution of feature points processed by FAST feature points extraction algorithm homogeneous.
After the feature points of the whole image are extracted by the FAST algorithm, the ORB-SLAM2 first uses the quadtree algorithm to divide the pyramid into smaller blocks. It filters the feature points of the image according to the corner response value of them. Generally speaking, the number of blocks divided by quadtree homogenization algorithm is related to the number of feature points, the more the number of feature points, the more the blocks divided. The area divided by quadtree algorithm is related to the number of original feature points. The more the number of feature points, the smaller the area. Firstly, the whole image is divided into four equal-sized nodes in the geometric center of the image. If there is no feature point within the node, delete the node. Then, judge the relationship between the number of deleted nodes and the size of N. If the number of remaining nodes is greater than N, stop segmentation, and select and retain the best quality feature points in each node according to the response value of corner points, and delete the remaining feature points. If the current number of nodes is less than N, the nodes with multiple characteristic nodes are further divided into four equal-sized groups.
In this paper, a proportional-based feature point homogenization algorithm is adopted. The experimental results show that the algorithm combined with the adaptive information entropy method has better performance.

C. PRINCIPLES OF INFORMATION ENTROPY
Information entropy [24], [25] is a measurement which describes the uncertainty of things. Entropy represents the disordered state of a system. In information theory, entropy is a measure of the degree of information disorder which is used to measure the uncertainty of the information in the image. A larger entropy value indicates a higher degree of disorder. In image processing, entropy reflects the information richness of the image. The amount of information contained in an image is expressed by information entropy. The entropy value of an M × N is defined as [26] where f (i, j) is the gray level at point (i, j) in the image, p ij is the probability distribution of gray level at point (i, j) and H is the entropy of the image. If there is a M × N block neighborhood with (i, j) as the center, then H is called the local entropy of the image. The local entropy reflects the richness of the texture information contained in the local image or the degree change in the image pixel gradient. The larger the local entropy value, the richer the texture information and the more obvious change in the image pixel gradient. Therefore, this kind of image blocks are retained as it is effective in extracting feature points in the process of FAST feature points extraction algorithm. On the other hands, the lower the local information entropy value, the less obvious the pixel gradient changes and less texture information. As results, the image blocks are removed because of the worse effect of extracting feature points. Information entropy can reflect the discreteness, noise, or signal distribution of pixels in each grayscale image, and also reflect the richness of image information. Information entropy has the ability to resist geometric deformation. The shape of the image is displayed by the pixels in different areas of the distribution. The information entropy reflects the total gray level dispersion of the window image, but it cannot reflect the specific distribution of a single pixel. When the image object is locally deformed geometrically, the statistical characteristics of the pixels remain unchanged, and its entropy value remains unchanged. Therefore, the information entropy value has a good resistance to geometric deformation. Based on the above analysis, the information entropy is suitable for the combination of ORB feature point extraction algorithms to improve the image mismatch problem under the conditions of image blur, illumination change, image rotation, affine transformation.

III. ALGORITHM IMPROVEMENT A. FLOW OF THE AIE ALGORITHM
This paper proposes a detection algorithm of ORB feature points based on information entropy. The flow diagram is shown as in Fig. 2.
First, the grayscale image is input, and then the image pyramid is built to make the image scale invariant by creating 8 images of different scales. Image blocks are divided according to predefined sizes. The information entropy and threshold of each image block are compared layer by layer. If the entropy value of image blocks is greater than the threshold value, they are selected. If the entropy value of image block is greater than the threshold value, a lot of texture information will be generated, which will have a great impact on the features. If the entropy value of image block is less than the threshold value, the contribution of image block to feature point extraction and texture image information extraction is very small. Therefore, feature points are not extracted. Then, FAST feature points are extracted and homogenized, and BRIEF descriptors are calculated in the selected image blocks. The homogenization process is to prevent the feature points from being too dense and clustering, so that the algorithm is more robust. After that, the feature points are matched and the matching rate is calculated. If the matching rate is the highest, the matching rate, matching result and information entropy threshold are the outputs.The information entropy threshold is closely related to different scenes. In different scenes, we need to find the most suitable entropy threshold through repeated experiments. The reason is that the experimental results in different scenes are quite different and not universal. Therefore, the adaptive algorithm to obtain information entropy threshold is particularly important. If there is no adaptive information entropy method in image matching, the information entropy threshold must be set many times for matching calculation, and the good matching results cannot be obtained quickly.
In view of the above problems, an adaptive information entropy algorithm(AIEF) is proposed, which can automatically calculate and select the threshold with the highest matching rate in different scenes. The algorithm steps are as follows: a. Calculate the information entropy E: where p ij is the probability distribution of gray level at point (i, j) and E is the entropy of the image. b. Set the number of cycles C i : The step S p and the information entropy threshold F are initially set to certain values respectively, and S p is added in each cycle.
c. The image of each scale is divided into several image blocks and the entropy value of the image block E ij is calculated. The threshold value F is compared with E ij of all image blocks. And the image block whose E ij is greater than F is retained. That is, the condition for the image block to be retained is: After traversing all image blocks of one scale image, enter the image of the next scale for new traversal until the comparison of all image blocks of the entire pyramid's 8 scale images is completed.
d. Caculate the Correct Matching Rate CMR: where m is the number of the feature points after Random sample consensus (RANSAC), n is the number of feature points before RANSAC. Then reset the information entropy threshold F:

B. PROPORTIONAL-BASED HOMOGENIZATION ALGORITHM
At first, we try to improve the algorithm by combining the adaptive information entropy with the quadtree homogenization algorithm in ORB-SLAM2 algorithm, but we find that the correct matching rate is not improved compared with the original ORB-SLAM2. The purpose of the quadtree homogenization algorithm is to make the distribution of feature points as homogeneous as possible, but it will reduce the requirements for the quality of feature points at the same time.
In the AIEF algorithm, the local information entropy limits the application area of the FAST algorithm. It makes the extracted feature points have good quality within the application area. Therefore, by using the quadtree homogenization algorithm, some feature points with good quality will be deleted originally, while feature points with poor quality are retained. Based on the reasons above, a proportional-based homogenization algorithm is applied in the AIEF algorithm. Through experiments, N A feature points can be extracted to obtain higher resolution pixels when FAST algorithm was used. With N A as a parameter for setting the whole image pyramid, the number of feature points to be extracted from the nth layerN n can be expressed as The length of the image block is l and the width of the image block is w. Thus, the number of image blocks in each layer m n is The number of image blocks in the image of this layer is calculated, and then the number of feature points extracted in VOLUME 8, 2020 each image block is given by the following formula: k = N n m n (14) Comparing the number of feature points actually extracted in the image block R i (i ≤ 8 n=0 m n ) with k. And C is denoted as the number of image blocks whose number of feature points is actually no more than k using the FAST algorithm. When the number of feature points actually extracted in an image block is no more than k, all feature points are retained. Otherwise,k feature points are retained. After traversing the entire image pyramid, the number of unallocated feature points in the entire image pyramid is A, we can get: Then, A is evenly divided into image blocks,whose number of feature points A j ≥ k, Therefore, update k; Repeat the above operation until A is zero.

IV. EXPERIMENTAL RESULTS
In order to quantitatively analyze the image matching effect of the AIEF algorithm, the correct matching rate, extraction and description time, the number of feature points and matching number are used as evaluation criteria to comprehensively evaluate the performance of the algorithm. The correct matching rate is the ratio of the number of correct matches to the feature points selected by the algorithm after screening. Here the number of feature points filtered by RANSAC [29] is simply referred to as the number of feature points. As for the image matching effect, the larger the value of the correct matching rate, the better the matching effect is. The result is best when the cycle step of AIEF is set to 0.2 based on repeated experiments. Through experiments, we conclude that the AIEF algorithm solves the image mismatch problem under the conditions of image blur, lighting change, image rotation and affine transformation. It also improves the matching rate of feature points in these scenarios.

A. EXPERIMENTAL ENVIRONMENT
All datas in this paper are stored in a laptop computer, its operating system is Linux 16.04 64-bit, processor Intel (R) Core (TM) i5-7200U CPU @ 2.50GHz, 2712 Mhz, 2 cores, 4 logical processors. The running environment is CLion 2019 and opencv3.3.0, and the program is written in C++. Four groups of data in the Oxford standard data set were tested: Graf, bikes, boat and Leuven. In the experiment, the first two images in each image group were taken out for testing. The test images are show in Fig. 3.

B. ANALYSIS OF EXPERIMENTAL RESULTS
The images from Oxford dataset are tested by feature points matching algorithm from ORB-SLAM2, SIFT,SURF AND AIEF algorithm separately. Among them, SIFT, ORB and AIEF algorithm all extract 1,000 feature points, and then select the feature points with good quality after screening to match. The SURF algorithm cannot directly set the number of feature points, and can only change the number of extracted feature points by setting the value of the Hessian matrix [30].

1) ANALYSIS OF CORRECT MATCHING RATE
Random sample consensus (RANSAC) is an iterative method to estimate the parameters of the mathematical model from a group of observed data containing outliers. In visual SLAM algorithm, RANSAC algorithm is used to eliminate image mismatch. Here we use the change of feature points before and after the process of RANSAC algorithm to represent the correct matching rate of the algorithm. The results of the CMR are shown in Table 1. It can be seen from Table 1 that the matching rate of the AIEF algorithm is better than other algorithms. The AIEF is 2.75%, 13.12% and 25.48% higher than ORB-SLAM2, SIFT and SURF algorithm respectively on average. The matching rate of AIEF is also superior to the algorithm combined with Quadtree and entropy. The algorithm combined with Quadtree and entropy has not improved so much compared to ORB-SLAM2, and in most cases the matching accuracy obtained with ORB-SLAM2 are the same. That is because the information entropy has a pre-screening function to the feature points, the selected feature points have the information-rich characteristics, so the matching effect is more obvious.
Here, only the feature points matching effect diagrams of the first and second pictures in the Boat image group are shown in Fig. 4, where Fig. 4(a), Fig. 4(b), Fig. 4(c), Fig. 4(d) and Fig. 4(e) correspond to the matching effect diagrams of ORB-SLAM2, SIFT, SURF, the AIEF algorithm and the combination of quadtree and entropy algorithm respectively. Fig. 4 shows that the AIEF algorithm finally gets homogeneous distribution of feature points due to the use of proportional-based homogenization algorithm to segment feature points. It is even better than the combination of quadtree and entropy.

2) TIME OF EXTRACTION AND DESCRIPTION ANALYSIS
Due to the application of AIEF algorithm, some image blocks whose information entropy does not meet the threshold condition will be removed in advance when the FAST algorithm extracts feature points. Compared with other algorithms, the time of feature point extraction and matching in AIEF algorithm will be greatly shortened as the image of feature point extraction becomes smaller and smaller. The extraction and matching time of ORB-SLAM2, SIFT, SURF, AIEF algorithm and the combination of quadtree and entropy algorithm are tested by Oxford data set. The results are shown in Table 2:  It can be seen from Table 2 that the extraction and description time of the AIEF algorithm is shorter than that of other algorithms. Particularly, the operation time of the algorithm combined with quadtree and entropy is even longer than that of ORB-SLAM2 system. The AIEF algorithm allows good quality feature points to be retained, so the extracted feature points can be retained directly. Using the quadtree algorithm will lead to longer extraction time and therefore longer time of program operation. The AIEF algorithm uses VOLUME 8, 2020 the FAST algorithm to extract and retain the feature points of the image blocks after the deletion of adaptive information entropy more directly, which saves the running time.

3) ANALYSIS OF FEATURE POINTS MATCHING
Compared with ORB-SLAM2 system, there are some changes in feature point extraction and homogenization in the AIEF algorithm. The information entropy of image block with rich texture is high, and the feature points extracted from such image block have better quality.Using the proportional-based homogenization algorithm proposed in this paper, we can extract more and better quality feature points than the algorithm in ORB-SLAM2. In order to verify that the number of feature points and matching points extracted by our algorithm is greater than the number extracted in ORB-SLAM2 and the combination of quadtree and entropy algorithm, The different algorithms are tested with images from the Oxford dataset, as shown in Fig. 1.The results of the number of feature points and matching points are shown in Table 3 and table 4.  It can be seen from table 1 that although the correct matching rate of AIEF algorithm is higher than that of ORB-SLAM2 system, it is not much better. According to table 3, the results of ORB-SLAM2 algorithm and the combination of quadtree and entropy algorithm are almost the same. The average number of feature points extracted by the AIEF algorithm is 80 more than that of the other two algorithm, and the average number of correct matching is 70 more than that of the other two algorithm. Although the correct matching rate of AIEF algorithm is not much higher than that of ORB-SLAM2 algorithm, the number of feature points and matching points extracted by AIEF algorithm are much higher than that of ORB-SLAM2 algorithm.

V. CONCLUSION
In this paper, adaptive information entropy (AIE) is applied to image matching algorithm innovatively. The key point is to combine AIE with image homogenization algorithm. A feature point extraction algorithm based on adaptive information entropy, i.e., AIEF algorithm is proposed. The algorithm is applied to image matching, and the results show that the algorithm can solve the problem of image mismatch caused by image blur and rotation to a certain extent. Next, AIEF algorithm and SLAM algorithm are combined to optimize and improve the accuracy and robustness of SLAM algorithm. Firstly, AIEF algorithm calculates the information entropy of image block. Then, the image blocks with less information are removed and the feature points are extracted from the reserved image blocks. In addition, the direct homogenization algorithm replaces the original quadtree algorithm used in ORB-SLAM2. The algorithms are tested with Oxford dataset. The results show that AIEF algorithm is superior to ORB-SLAM2 algorithm in terms of processing time, number of feature points, number of correct matches and rate of correct matches. In addition, we will further study the application of the algorithm in different specific noise scenes, and try to apply it to practice.