Bi-Normal Mesh Smoothing Based on Vertex Feature

Aiming at the issue that mesh smoothing is hard to balance in terms of noise removing and feature preserving, in this article, we combine the facet normal representing global geometric features of the mesh with the vertex normal characterizing local details of the mesh and propose a bi-normal mesh smoothing method based on vertex feature selection (BNBVF). Firstly, the guided filtering algorithm is extended to calculate accurately the facet normal in the field of geometric processing. The key of this portion is computing the guided normal of facet by using a geometric neighboring patch with the most consistent normal. Then, an adaptive tensor voting method is employed to divide the vertices of the mesh into feature vertices and non-feature vertices. Thirdly, a method of the neighborhood facets clustering combining with the plane fitting is proposed to calculate the normal of feature vertex, and the weighting average of first-order neighborhood facets of the vertex is applied to compute the normal of non-feature vertex. Finally, the vertices of the mesh are updated iteratively by combining the geometric information of the facet normal and vertex normal to achieve mesh smoothing. Experimental results demonstrate that the superior performance of our proposed algorithm to state-of-the-art approaches in feature preserving and error reducing.


I. INTRODUCTION
Industrial computed tomography (CT) can clearly, accurately and intuitively display the internal structure, composition, material and defect condition of the detected object in the form of two-dimensional sectional image or threedimensional image without damage to the detected object. The basic principle is that x-ray or γ -ray with certain energy and intensity is used to irradiate the detected object, and then the attenuation law and distribution of energy in the detected object are calculated. Finally, Industrial CT data of the detected object is obtained by using computer information processing and image reconstruction technology. However, it is affected by some factors such as instrumentation, ray source noise, reconstruction algorithm and human factor during data acquisition, storage and transmission, which makes the noise information inevitably existing in the triangular The associate editor coordinating the review of this manuscript and approving it for publication was Varun Gupta. mesh reconstructed by industrial CT data [1], [2]. The presence of these noises not only reduces the visualization of the mesh, but also brings about some troubles for subsequent processing. Although the existing mesh smoothing methods have achieved satisfactory results, there remains still a challenging issue that non-uniform sampling and multi-scale triangular mesh reconstructed by industrial CT data can effectively preserve geometric detail of the mesh while smoothing.
The current mesh smoothing methods include mainly isotropic smoothing and anisotropic smoothing. The Laplacian method [3] has become a representative of isotropic methods and always favored by researchers because it is fast, easy to implement and widely used. Taubin et al. [4] proposed a two-step Laplacian operator to extend Laplacian method for mesh smoothing. Desbrun et al. [5] extended two-step Laplacian method by geometric flow simulation and applied it to smooth the non-uniform mesh. Meyer et al. [6] improved the geometric flow simulation further and applied it to preserve anisotropic feature. Kim and Rossignac [7] VOLUME 8,2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ designed a low-pass/high-pass filter frame with enhancement and attenuation options for smoothing the mesh. Indeed, these isotropic smoothing methods can achieve stable results, however some local details of the mesh are inevitably filtered out due to the inherent characteristics of isotropic filtering. Compared with the isotropic mesh smoothing method, the anisotropic smoothing method can better preserve the geometry and detail features of the mesh while smoothing the mesh. Fleishman et al. [8] extended firstly the bilateral filtering algorithm that is well filtering effect in the field of image processing, and applied it to smooth the mesh by means of a fast one-step iterative scheme. However, this method cannot always accurately preserve the details of the mesh. Jones et al. [9] employed a two-step non-iterative scheme to consider normal smoothing and vertex updating as a global optimization problem, which enhances the retention of features to a certain extent. However, the algorithm is slow. Zheng et al. [10] used bilateral filtering to process the facet normal of the mesh, and then updated the vertex position according to the facet normal. Solomon et al. [11] modified bilateral filtering to deal with the facet normal, and combined with the mean shift method to implement a framework for mesh smoothing, which improves the speed and smoothing effect of the bilateral filtering algorithm to some extent. Nevertheless, the framework is not effective to all meshes. In recent years, a two-stage method (or its variants) [12]- [17] that the facet normal is firstly calculated and then the vertices are iteratively updated according to the facet normal is proposed to achieve mesh smoothing, and it can better preserve the geometric and detailed features of the mesh with certain noise. However, the method cannot effectively eliminate the mutual influence of different geometric structures, resulting in the fuzzy features. The basis of this method is that the facet normal is better than the vertex normal in describing the global geometric variation of the mesh, but the vertex normal can better describe the local detail features of the mesh. Therefore, the non-uniform and multi-scale mesh smoothing only based on geometric information of the facet normal can result in the blurring of the detail features, shape distortion or vertex offset of the mesh [18]- [20]. Moreover, Wei et al. utilize surface normal filtering to avoid the loss of geometric details in bas-relief modeling [21], [22].
In addition, sparse optimization and surface reconstruction for mesh de-noising have increasingly attracted more attention of researchers in recent years. Several novel l 0 and l 1 sparse optimization method [17], [17], [23]- [25] have been proposed to eliminate robustly and reliably noises while preserving features. However, it is crucial to estimate properly the differential geometric properties of mesh in these methods. Some 3D-surface reconstruction methods [26]- [28] have been presented to remove noisy while recovering feature. Whereas, the effective structure detection method for estimating properly surface normal is essential due to most meshes usually contain complicate structure. Though the above-mentioned methods can remove the noisy from mesh while preserving features to some extent, there is still great space to improve the effect of mesh smoothing. In this article, the guided filtering is extended to process the facet normal of the mesh, combining the vertex normal calculated by the vertex feature, a novel bi-normal method for mesh smoothing is proposed. The scheme of the presented method is listed in FIGURE 1.
The work foci of this article is three-fold: (1) A novel mesh smoothing method that considers both facet normal and vertex normal is developed.
(2) The authors extend the guided filtering to calculate accurately the facet normal according to the patch whose consistency with the facet is best.
(3) A method of the neighborhood facets clustering combining with the plane fitting is proposed to calculate precisely the normal of feature vertex.

II. COMPUTING THE FACET NORMAL FIELD BASED ON GUIDED FILTERING
In consideration of the field of image processing, the guided filter is better than the bilateral filtering in feature retaining and arithmetic speed. Inspired by the good performance of the bilateral filtering in the field of mesh smoothing, this article introduce the guiding filtering to the field of geometry processing for mesh smoothing.

A. THE GUIDED GILTERING
Guided filtering is a new image filtering method, and its output is a local linear transformation of the guided image. The guided filtering has good edge retention performance similar to bilateral filtering and does not cause gradient inversion. Moreover, if the guidance image is appropriate, the output of the guided filter has more structural information than the input image. Because the guided filtering is simple, effective and the features are kept well, it has been widely used in image smoothing/enhancement, flash/non-flash image, image mapping/gradation processing, de-blurring, up-sampling, optical flow estimation, image segmentation and illumination and rendering [29].
The key to the guided filtering is that can be regarded as a local linear model between the guidance image and the output image [29], [30]. If the output of the guided filter is q and the guidance image is G, the linear relationship between the output and the guided image can be defined as follows: where, ω k is a rectangular window whose central pixel is k, and its radius is r. a k and b k are linear coefficient in window ω k , and respectively defined as: where, p k is the mean of input image p in window ω k . According to (1), we can know that the output image must have edge information as long as the guidance image G has edge information, because ∇q is equal to a∇G.

B. FACET NORMAL FILTERING
Compared with vertex filtering, normal filtering can describe the geometric features of the mesh more conveniently and accurately. For example, the sharp edge can be described by the larger error of normal between two incident faces. The outward unit normal of facet f i in the triangular mesh can be defined as: where, v i 1 , v i 2 , v i 3 are respectively the coordinates of each vertex of facet f i . According to (1), the guided filtering output of the unit normal of facet f i can be expressed as: where, where, g i is a guided normal of facet f i , g i denotes the mean of the guided normal that come from neighborhood facets of f i , n i indicates the mean of the normal that come from neighborhood facets of f i , ω is the number of facets in the neighborhood patch, N i represents the first-order geometric neighborhood of facet f i . The geometric neighborhood consists of the largest set of faces containing the facet f i and satisfies the following two conditions. (1) The normal angle between each facet belonging to N i and facet f i is less than or equal to β which is specified by user, that is, (2) A single connection is formed by sharing vertices between adjacent facets of N i . For a non-uniform mesh, the geometric neighborhood includes all neighborhood facets with similar normal, which can achieve better filtering output.

C. CONSTRUCTING THE GUIDED NORMAL OF FACET
In the guided filtering, the closer the input is similar to the original signal, the better the filtering output. Likewise, the closer the guiding normal to the facet normal is, the better the filtering effect of the mesh is. Therefore, the key to the quality of the filtered output is the construction of the guided normal.
If there is no noise in the mesh, the normal n i of the facet itself is a good guided normal. If the noise in the mesh is large, the normal of the facet itself cannot correctly describe the curve surface of the mesh, and using it as the guided normal may bring about erroneous results. Reference [14] proposed that a triangular mesh could be divided into a number of small patch composed of a plurality of facets with similar normal. For each facet f i , we can find a patch P that possesses the best consistency among all patches containing the facet f i . Therefore, we employ the normal of facets in patch P to calculate the guided normal of facet f i . All patches that contain facet f i can be represented by a patch candidate set where P k is a union patch composed of all facets that share a vertex with any facet f k of the mesh.
The normal consistency H(P) of each candidate patch P ∈ P set (f i ) is calculated as follows: where, D(P) denotes the maximum distance between the normal of two facets in patch P, and S(P) represents the relative saliency of the edges in the patch.
where, E P stands for the set of all inner edges in patch P, ϕ(e j ) is the saliency of the inner edge and it can be calculated by the normal of two adjacent facets f j 1 and f j 2 (e j represents the common edge of facet f j 1 and facet f j 2 ).
It can be seen by above (9) and (10) that the smaller D(P), the more similar the normal of all facets in the patch are, the smaller S(P), the more similar the saliency of all the inner edges in the patch. Therefore, the smaller the normal change of each facet in patch P, the smaller the consistency function H(P).
The guided normal of facet f i can be calculated by the weighting average of all facets of patch P * . Where P * ∈ C (f i ) and H(P * ) = min{H(P), P ∈ P set (f i ) P}.
where, ω f j = decreasing with respect to the distance between c i , c j , c i , c j present the center of facet f i , f j respectively.

III. CONSTRUCTING THE VERTEX NORMAL FIELD OF THE MESH
A. THE VERTEX CLASSIFICATION BASED ON TENSOR VOTING 1) THE TENSOR VOTING The voting tensor of vertex v i of facet f j in the triangular mesh can be regarded as the weight sum of the neighborhood triangle normal covariance matrix V f v [13], [31], [32]. If the unit normal of facet f j is n f j = (a, b, c), then V f v can be expressed as: Tensor voting T v i of vertex v i can be defined as: where, N f (v i ) represents the first-order neighborhood triangle of vertex v i , ω f j denotes the weight that is calculated by the area ratio of the neighborhood triangle and the distance between vertex v i and each triangle of N f (v i ).
where, S f j stands for the area of facet f j , max(S N f (v i ) ) denotes the max triangle area of N f (v i ), c f j is the center of triangle f j , and v i is the coordinate of vertex v i .

2) THE VERTEX CLASSIFICATION
Reference [32] proposed a vertex classification method based on the eigenvalues of normal tensor of the vertex, which can divide the vertices of the mesh into plane vertices (nonfeature vertices), sharp edge vertices and corner vertices (feature vertices). However, the method can only detect the feature vertices of the high-quality mesh because that employs only the normal information of the vertex first-order neighborhood facets to calculate the vertex normal tensor, no considering the scale information. There may be exist false detection and missed detection of feature vertices when the method is employed to deal with a noisy and non-uniform sampling mesh, such as the mesh reconstructed by industrial CT. Therefore, this article adopt the adaptive tensor voting method proposed in Reference [13] to classify the vertices. This adaptive tensor filtering can not only filter the noise, but also preserve the detailed features of the mesh and achieve a good balance between feature vertices and non-feature vertices. The threshold i for distinguishing vertex types can be defined as follows: where, λ 1 , λ 2 , λ 3 denote the eigenvalues of the tensor voting respectively, and satisfy λ = (λ 1 + λ 2 + λ 3 )/3. If the vertex is a feature vertex, then i = 0. The pipeline of the vertex classification is as follows.
Step1: r m (m = 1) that is the first-order triangle neighborhood of vertex v i is set to initial scale.
Step2: Calculate the value of r m i and compare it with 1 , 2 . If r m i ≥ 2 , then vertex v i is a feature vertex and the neighborhood expansion is stop; if vertex v i is a non-feature vertex and the neighborhood expansion is stop too; otherwise, step3 is executed.
Step3 The normal tensor voting at scale r m can be defined as follows: (17) where, N r m (v) represents new added facets at scale r m , and weight coefficient ω f j can be defined as: where, m indicates the order of the neighborhood of vertex v i , max(S N rm (v) ) denotes the max area of the facet which comes from m-order neighborhood of vertex v i , S f j represents the area of each facet in m-order neighborhood of vertex v i , max m ( c f j − v i ) stands for the max distance between vertex v i and the center of the facet in m-order neighborhood. In addition, the purpose of adding m to the numerator of the exponent exp(·) is to ensure that the inner ring neighborhood of vertex v i is more influential than the outer ring neighborhood in calculating tensor voting.

B. CALCULATING THE NORMAL FIELD OF THE VERTICES 1) CALCULATING THE NORMAL OF THE NON-FEATURE VERTEX
The non-feature vertex whose change of geometric feature around it is smaller, and the normal of vertex is roughly consistent with the normal of the facet in neighborhood. Therefore, the normal of non-feature vertex can be calculated by the weighting average of the first-order neighborhood facets of vertex. where, is the weighting factor of the first-order neighborhood facet, S f j indicates the area of facet f j , and n f j represents the normal of facet f j .

2) CALCULATING THE NORMAL OF THE FEATURE VERTEX
Because the feature vertex normal of the mesh has a large difference from the normal of its neighboring facets, there is a big error if the weighting average of the neighborhood facet normal is still employed to calculate the feature vertex normal. In this article, the method of the neighborhood facets clustering combining with the plane fitting [6], [7], [11], [12] is utilized to calculate the normal of feature vertex accurately.

a: NEIGHBORHOOD FACETS CLUSTERING
The neighborhood facets clustering is the clustering of neighborhood facets with uniform geometric features, such as, normal similarity or smooth variation, around the feature vertex. References [12], [13] employed an energy function to describe clustering as an optimization problem. (20) where, k indicates the sorts of the neighborhood facets, M i denotes a certain neighborhood, ρ(x) is a density function that is a constant, n i represents the normalize weighting average of the normal of all facets in M i . According to formula (20), the energy function E regards the total offset of all facets in the class as the normal representative of the class. If the normal of all vertices in the facet are the same and s j = f j ρ(x)dx is the area function of facet f j , then the discrete form of E can be expressed as: VOLUME 8, 2020 where, n j indicates the unit normal of facet f j . If ρ(x) = 1, then s j is the area of facet f j . Neighborhood facets clustering of the feature vertex has been realized after formula (21) is optimized. For detailed optimization solution process, please refer to reference [12].

b: CALCULATING THE NORMAL OF THE FEATURE VERTICES
Step1: The least squares method [33] is employed to fit a cubic surface for the vertices of each clustered patch, and the fitting formula is as follows: f (x, y) = Ax 3 + By 3 + Cx 2 y + Dxy 2 + Ex 2 + Fy 2 + Gxy + Hx + Iy + J (22) Step2: The facet with the best fitting is searched in each type fitted cubic surface.
Step3: The normal of the feature vertex is calculated by the normal of facet with the best fitting in each type fitted cubic surface. (23) where, ω f j represents the weighting factor of facet f j with the best fitting, n j indicates the normal of facet f j with the best fitting. Step4: Traverse each feature vertex of the mesh and repeat step1∼step3 to generate the normal field of the feature vertices.
The normal of the facet that is the most representative of geometric feature information of the vertex is employed to calculate the normal of the feature vertex, which is more accurate than the weighting average of the normal of the firstorder neighborhood facets.

IV. MESH SMOOTHING BASED BI-NORMAL FILTERING
The facet normal and vertex normal of the mesh describe global geometric features and local details of the mesh respectively. The geometric feature of these two normal is particularly repetitive sometimes, and even almost same in the flat region of the mesh. However, the mesh is mostly segmented smooth and the geometric features of the facet normal and vertex normal are no longer similar in the reverse engineering field based industrial CT. In this case, only the facet normal or the vertex normal is employed to smooth the mesh, which results in blurred details and shape distortion of the mesh. Therefore, in this article, a novel mesh smoothing method based on bi-normal filtering is proposed by combining the vertex normal with the facet normal. The presented method can better preserve the geometric detailed features of the mesh, and the specific steps are as follows.
Step1: The guided filtering is extended to calculate accurately the facet normal field of the mesh.
Step2: The tensor voting method is used to classify the vertices of the mesh into non-feature vertices and feature vertices.
Step3: The vertex normal field of the mesh is respectively calculated according to the vertex type.
(1) If the vertex is a non-feature vertex (i.e. plane vertex), the vertex is updated via the facet normal n * i of the mesh.

v
where, v (t) i is the value of vertex v i at the t-th iteration, N i indicates the first-order neighborhood facet set of the vertex v i , and c (2) If the vertex is a feature vertex (i.e. sharp edge vertex or corner vertex), the vertex is updated by combing the facet normal n * i with the vertex normal (25) where, n * j denotes the facet normal of the first-order neighborhood facet processed by the guided filtering, n v i indicates the normal of the vertex, λ stands for the ratio factor and the range of value is 0 ≤ λ ≤ 1.
Step5: Traverse all vertices of the mesh and repeat step1∼step4 to update each vertex.
Step6: If the error of between smoothed mesh and the initial mesh or the error between the current smoothed mesh and the former smoothed mesh is less than the setting value, the algorithm is stopped. Otherwise, the algorithm flow return to step1 to continue processing.

V. EXPERIMENTAL RESULTS AND ANALYSIS
To validate the effectiveness in both noise eliminating and feature retaining of the proposed method, the method is compared against the classical and state-of-the-art methods on some meshes, such as, Laplacian (LAP) [3], bilateral filtering (BF) [8], bilateral normal filtering (BNF) [10], joint bilateral filtering (JBF) [14], bi-normal filtering based on optimization (BNFBO) [12], and PcFilter [34]. Since each smoothing method involves many parameters and some of the parameters are changed, the parameters are set according to the recommendations of the original literature, and the experiment is repeated several times to select the best result for comparison.
In order to demonstrate the versatility of the proposed method, the big Gaussian noise is added artificially to the Fandisk mesh and the holder mesh. FIGURE 2 and FIGURE 3 show the comparative results of the noisy Fandisk and the noisy holder in different smoothing methods. BF is not effective in dealing with such mesh with largescale noise, and can only remove part of the high-frequency noise. BNF and BNFBO can better eliminate noise, but the edges and detailed features of the mesh need to be improved. The smoothing effects of PcFilter and BNBVF method both are good, and BNBVF method is better in terms of detailed feature-preserving and has the best visualization effect. FIGURE 4 shows the results of the cylinder head mesh reconstructed by industrial CT image in different smoothing methods. According to Figure 4, the visualization effects of the cylinder head mesh in different smoothing methods can be observed clearly. The smoothing result of LAP is too smooth. The smoothing results of BF and JBF contain still some noises, however, the features can better be preserved. BNFBO, PcFilter, and BNBVF not only can remove the noises of the mesh, but also can better preserve the detailed features of the mesh. In addition, the smoothing results of these three methods are the best. FIGURE 5 show that the visualization effect of the carburetor mesh smoothed by various methods. The detailed features of the mesh smoothed by LAP are lost, and the surface of the mesh is too smooth. The method of JBF improves the ability of feature preserving to some extent, however the surface of the mesh is still too smooth. Part of the noises of the mesh is not removed in the methods of BF and BNF, and the feature preserving of the mesh in BNF is better. Both BNBVF, BNFBO and PcFilter can better retain the detailed features of the mesh, and the smoothing results are perfect too.
To objectively compare the proposed and existing smoothing methods, the error information of the mesh smoothed by different methods is counted, the specific information is as follow: minimum bounding box size S Box , surface area error E sur , volume error E vol , maximum distance of the vertex D max and average distance of the vertex D ave . Table 1 shows the related information of the original cylinder head mesh and carburetor mesh. The error information of the meshes smoothed by different methods are shown in Table 2 and Figure 6. According to the data in Table 2, the meshes smoothed by different methods including BNBVF, BNFBO and PcFilter, can better approximate the original mesh, as well as the volume error and surface area error are smaller. Whether the carburetor mesh or the cylinder head mesh, the errors of PcFilter are the smallest, and BNBVF's is similar. However, the maximum distance and average distance of the vertex of BNBVF are the smallest. A more intuitive performance comparison of different smoothed mesh is demonstrated in FIGURE 6. In addition, the energy function is employed to express the vertex update under PcFilter, which brings about the slow operation speed because of the complicated optimization process.

VI. CONCLUSION
In allusion to the defect that the mesh is hard to balance in terms of mesh smoothing and feature preserving, this article put forward a bi-normal mesh smoothing method by combining the facet normal representing global geometric features of mesh with the vertex normal characterizing local details of mesh. Compared with the existing mesh smoothing methods, the proposed method can better preserve the geometric detail features of the mesh while removing the noising, and the error between the smoothed mesh and the original mesh is smaller. Therefore, the mesh smoothed by the presented method can more accurately approximate the original mesh. The shortcoming is that there are many parameters involved in the method, and these parameters need to be re-adjusted to obtain better smoothing results for different meshes. In addition, the scale factor for evaluating the contribution of the facet normal and the vertex normal has a greater impact on the results. Therefore, for different meshes, how to adjust automatically these parameters to obtain better smoothing results will be the focus of subsequent work to study.