Dimensionality Reduction Method for 3D-Handwritten Characters Based on Oriented Bounding Boxes

Dimensionality reduction of 3D-handwritten characters can be problematic because of random mirror rotations and angle rotations which appear after using the traditional algorithms. In order to overcome the above drawbacks of the traditional methods, this study proposes a new algorithm for dimensionality reduction of 3D-handwritten characters based on oriented bounding boxes. First, we get a 3D discrete point set <inline-formula> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> and generate a 3D trajectory. Then, we apply an oriented bounding box model and determine the projection surface. Next, we perform three coordinate transformations, including (1) pre-transforming the 3D discrete point set <inline-formula> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> into the projection point set <inline-formula> <tex-math notation="LaTeX">$T_{1}$ </tex-math></inline-formula>, (2) converting <inline-formula> <tex-math notation="LaTeX">$T_{1}$ </tex-math></inline-formula> to a two-dimensional point set <inline-formula> <tex-math notation="LaTeX">$T_{2}$ </tex-math></inline-formula> which has solved the problem of mirror rotation, and (3) converting <inline-formula> <tex-math notation="LaTeX">$T_{2}$ </tex-math></inline-formula> to a dimensionally reduced point set <inline-formula> <tex-math notation="LaTeX">$T_{3}$ </tex-math></inline-formula> which has solved the problem of angle rotation. Finally, we obtain a dimensionally reduced image without mirror and angle rotations. The experimental results confirm that the proposed method can not only obtain a better visual dimensionally reduced image, but also has a higher recognition rate than the conventional ones.


I. INTRODUCTION
3D-Handwritten character recognition is one of important method to realize non-contact human-computor interaction. Leap motion can provide 3D point cloud to track 3D finger movements with 0.01 millimeter accuracy. Based on Leap motion, many algorithms of 3D-Handwritten character recognition have been designed [1]- [3], which proceed the 3D-Handwritten character recognition directly on the 3D track. However, high dimensionality has brought great pressure to the transmission and storage [4]. Moreover, three are still mature algorithms of 2D-Handwritten character recognition. As a result, another way to recognize the 3D-Handwritten character is transforming 3D-Handwritten character into 2D character by dimensionality reduction method firstly, and then 2D character containing the equivalent information of 3D-Handwritten character is recognized by the 2D character recognition method to finish the recognition of 3D-Handwritten character.
The associate editor coordinating the review of this manuscript and approving it for publication was Chunming Gao.
Dimensionality reduction is an important process for retaining as much high-dimensional data as possible in a low-dimensional space. The existing dimensionality reduction algorithms can be linear or nonlinear. Traditional linear dimensionality reduction algorithms include Principal Component Analysis [5] (PCA) which is to select the lowdimensional space with the largest variance as the projection space and Linear Discriminant Analysis [6] (LDA) which is to maximize the distance between classes and minimize the distance within the class. The dimensionality reduction algorithms for nonlinear data can be divided into those retaining global features and retaining local features. Among them, the algorithms that preserve local features include Locally Linear Embedding [7] (LLE), which is an unsupervised learning algorithm that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs. Laplacian Eigenmaps [8] (LE), which is that the points that are related are as close as possible in the space after dimension reduction and it can reflect the internal manifold structure of the data. Local Tangent Space Alignment [9] (LTSA),which uses the local tangent space to represent VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ the local geometry of the basic manifold structure, Hessian Eigenmaps [10] (HE), which adapts the weights in LLE to minimize the Hessian operator. And Locality Preserving Projections [11] (LPP), which arises by solving a variational problem that optimally preserves the neighborhood structure of the data set. From the set of nonlinear algorithms that retain the global features, several algorithms use kernel technology such as Kernel Principal Component Analysis [12] and Kernel Fisher Discriminant Analysis [13], which can be seen as nonlinear versions of classic algorithms such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). Another type of dimensionality reduction algorithms is based on distance retention and includes Multiple Dimensional Scaling [14] (MDS) which keeps the distance between the original spatial samples and the distance in the lowdimensional space as much as possible, Isometric Mapping [15] (Isomap) which is a non-iterative globally optimized dimensionality reduction algorithm that requires the geodesic distance to remain constant before and after dimensional reduction, and Diffusion Maps [16] which is dimensionally reduced by diffusion processes to remove redundant information.
However, applying the existing dimensionality reduction algorithms to process 3D-handwritten characters has the disadvantage that the 2D-characters may be subject to mirror rotation and angle rotation. Moreover, this problem will affect the recognition rate for 3D handwritten characters. Figure 1 shows the images with mirror rotation (b) and angle rotation (c), compared to the original image (a).
In order to solve the problems of random appearance of mirror rotation and angle rotation, which are caused by dimensionality reduction of 3D handwritten characters, we propose a new dimensionality reduction algorithm based on oriented bounding boxes.
In this study, we acquire the 3D coordinates of handwritten characters by recording the 3D trajectory of the moving finger in the projection region of Leap Motion (projection region is a inverted pyramid region centered on Leap Motion, with an effective range from 26 nm to 600 nm above the device), define the coordinates of 3D-handwritten character as a 3D discrete point set T , and generate a 3D track of handwritten characters according to T . Then, an oriented bounding box that surrounds the 3D-handwritten character's discrete points is determined. Next, the projection surface is obtained by ascertaining the normal vector of the largest area in the oriented bounding box and the geometric center of the 3D discrete point. Next, we pre-transform the 3D discrete point set T into a projection point set T 1 and establish a 2D Cartesian coordinate system X O Y . Following this, by converting the projection point set T 1 to a two-dimensional point set T 2 and converting the 2D point set T 2 to a dimensionally reduced point set T 3 , we can solve the problem of mirror and angular rotations in dimensionally reduced images. Finally, a reduced-dimension image in a fixed direction is the output.
The remainder of the paper is organized as follows. Section 2 presents the proposed algorithm. The experimental results and a performance comparison with other state-of-the-art parameter estimation approaches are reported in Section 3. Section 4 concludes our study.

II. PROPOSED METHOD
The flowchart of the proposed method is shown in Figure 2. It is comprised of the following seven steps.
The proposed method is mainly aimed to solve the problem of random mirror and angle rotation of 2D-handwritten characters obtained by traditional dimensionality reduction methods. In our case, converting the projection point set T 1 to a 2D point set T 2 solves the problem of mirror rotation, and converting the 2D point set T 2 to the dimensionality reduction point set T 3 solves the problem of angular rotation.
Step x (Get 3D Discrete Point Set T and Generate 3D Trajectory): The proposed algorithm is based on the 3D coordinate system XYZ built into the Leap Motion somatosensory controller, which is defined as the initial 3D coordinate system of handwritten characters.
The characters are written by moving the index finger within the detection region of the Leap Motion. The 3D point set obtained by moving the index finger is represented by the set T , T ={T i (x i , y i , z i )}, i=1,2,. . . ,n. T i is the spatial coordinate of the i-th discrete point. n is the number of discrete points obtained by Leap Motion. Wherein x i , y i , z i respectively represent the coordinate values of the X-axis, the Y-axis, and the Z-axis of the discrete point in the initial 3D coordinate system XYZ. Then, 3D discrete points are sequentially connected to obtain a 3D trajectory of handwritten characters.
Setp y (Establish an Oriented Bounding Box Model): After obtaining the 3D discrete points of the handwritten characters, we can get an oriented bounding box surrounding all 3D discrete point sets. The parameters of the oriented bounding box are given below.

A. DIRECTION OF THE ORIENTED BOUNDING BOX
Since 3D discrete point set T can be obtained, so the mean T c (x c , y c , z c ) of all discrete points can be calculated, wherein x c , y c , z c respectively represent the mean coordinate values of the X-axis, the Y-axis, and the Z-axis of the center point of the discrete point set T in the initial 3D coordinate system XYZ. The average value can be calculated as (1), n is the number of input discrete points.
The covariance matrix C can be calculated as (2), as shown at the bottom of the next page. The discrete points need to be normalized of data, that is ( Since the covariance matrix C is a 3 × 3 symmetric matrix, there are λ and non-zero vectors α to satisfy (3). λ is feature value in C, and α is feature vector of C corresponding to the feature value. The eigenvalues λ =(λ 1 , λ 2 , λ 3 )and the corresponding eigenvectors α = (α 1 , α 2 , α 3 ) can be calculated as (3), wherein each feature vector is 3×1 vector.
Because the eigenvectors of the symmetric matrix C are orthogonal, the three eigenvectors(α 1 , α 2 , α 3 )only need to be normalized to obtain the base vector (v 1 , v 2 , v 3 ) as (4).
The three-axis direction of the oriented bounding box is defined by the base vector (v 1 , v 2 , v 3 ) and a 3×3 matrix consisting of (v 1 , v 2 , v 3 ) as the rotation matrix R, which is the rotation matrix R of the oriented bounding box.

B. VERTICES OF THE ORIENTED BOUNDING BOX
The maximum and minimum values of the X-axis, Y-axis, and Z-axis can be determined based on the maximum and minimum values of all discrete points T on each coordinate (2) VOLUME 8, 2020 axis of the initial coordinate system, that is x max , x min , y min , y max , z min , z max . The following shows eight points defined according to the above six coordinate values: A bounding box composed of the above eight points can surround all discrete points of 3D-handwritten characters, but this box cannot represent the direction of the characters. To obtain an oriented bounding box, it is necessary to multiply the eight points A 1 , B 1 , C 1 , D 1 , E 1 , F 1 , G 1 and H 1 by the rotation matrix R to obtain the directional vertices A, B, C, D, and E, F, G, H .
Let us take point A as an example, where A 1 is a 1×3 matrix, R is a 3×3 matrix, and A 1 is multiplied by R to obtain 1×3 matrix, that is, the 3D coordinates of the vertex A in the oriented bounding box. The 3D coordinates ofB, C, D, E, F, G, and H can be obtained similarly in (5).

C. SIDE OF THE ORIENTED BOUNDING BOX
We project the discrete point set T to the rotation matrix R (v 1 , v 2 , v 3 ), and then we find the projection interval of all the discrete points to the three directions, that is, the length of three projection intervals are the three sides of oriented bounding box. The maximum and minimum values of the ith discrete point T i projected along the respective directions are calculated as (6) and (7).
In (6), u 1 , u 2 and u 3 are the maximum values of point T i along the direction of v 1 , v 2 and v 3 . In (7), l 1 , l 2 and l 3 are the minimum values of point T i along the direction of v 1 , v 2 and v 3 . Then, three side length of the oriented bounding box is u 1 − l 1 , u 2 − l 2 and u 3 − l 3 .
The center point of the oriented bounding box is denoted as c, which is shown in (8).  Step z (Determine the Projection Surface MNPQ): The projection surface MNPQ is determined based on the parameters of the oriented bounding box. The normal vector of the projection surface MNPQ is the same as the normal vector of the largest area in the oriented bounding box. The projection surface MNPQ passes through the center point T c of the discrete point set T .
In Figure 3, MNPQ is defined. The connection order of MNPQ is the same as the connection order of ABCD.
Step { (The 3D Discrete Point Set T is Pre-Transformed Into the Projection Point Set T 1 ): The set of projection points is represented by a set T 1 , T 1 ={T i (x i , y i , z i )}, i = 1, 2, . . . , n. T i is the coordinates of the i-th projection point. Wherein x i , y i , z i respectively represent the coordinate values of the X-axis, Y-axis, and Z-axis of a certain projection point T i (x i , y i , z i ) in the initial 3D coordinate system XYZ.
The process of this step is to project the set of discrete points T onto the projection surface MNPQ to obtain the projection point set T 1 , that is, Figure 4 shows the projection point T i (x i , y i , z i ), i = 1, 2, . . . , n is converted to the i-th projection point T i (x i , y i , z i ), i = 1, 2, . . . , n.
The process of pre-conversion is derived as follows. Assume that the normal vector of the projection plane MNPQ is n (a, b, c), and the projection plane passes T c (x c , y c , z c ). Projection point T i can be calculated according to projecting the discrete point T i (x i , y i , z i ) onto MNPQ, as shown from (9) to (11).
The definition of MNPQ is as (9): Since T i is over the MNPQ, there is the following (10): Because of T i T i − → n , Substituting (11) into (9): In (12), the T i is obtained after a discrete point T i is projected onto the projection surface MNPQ. According to the above method, we can calculate the projection point sets T 1 corresponding to the discrete point set T .
As shown in Figure 5(a), the 3D trajectory of the handwritten characters and the oriented bounding box is obtained. In Figure 5(b), the projection point set T 1 obtained by projecting 3D discrete point set T to MNPQ is determined.
Step | (Establish 2D Cartesian Coordinate System X O Y ): Establishing a 2D Cartesian coordinate system is divided into two steps. First, we project points M , N , P onto the XOY plane in the initial 3D coordinate system to get the point M 1 , N 1 , P 1 , and it is judged whether the points M 1 , N 1 , P 1 are clockwise or counterclockwise. Then, we establish a 2D Cartesian coordinate system based on two different situations.
If d < 0, as shown in Figure 6(a), M 1 , N 1 and P 1 are counterclockwise. Then, we establish a 2D Cartesian coordinate system X O Y . The positive direction of the Y axis is the direction of the point M to the point Q, and the positive direction of the X axis is the direction of the M to N .
If d ≥ 0, as shown in Figure 6(b), M 1 , N 1 and P 1 are clockwise. Then, we establish a 2D Cartesian coordinate system X O Y . The positive direction of the Y axis is the direction of the point M to the point N , and the positive direction of the X axis is the direction of M to Q.
Step } (The Profection Point Set T 1 is Converted to 2D Point Set T 2 ): The 2D point set T 2 is defined as follows: Here, T i is the i-th 2D point. Wherein x i , y i respectively represent the coordinate values of the X'-axis and Y'-axis of a certain projection point T i (x i , y i , z i ) in the coordinate system X O Y . In this step, the projection point set T 1 is converted to a 2D point set T 2 , that is, the projection point T i (x i , y i , z i ) is converted to the i-th 2D point T i (x i , y i ), where i = 1, 2, . . . , n.
T i (x i , y i ) is defined according to whether M 1 , N 1 and P 1 are clockwise or counterclockwise. If M 1 , N 1 , and P 1 are connected counterclockwise, we establish a 2D coordinate system X O Y as shown in Figure 7, p i and q i mean the corresponding lengths of 2D coordinates. VOLUME 8, 2020  The coordinate value of the i-th 2D T i is as shown in (14).
If M 1 , N 1 , and P 1 are connected clockwise, and the i-th 2D T i is shown as (16).
The calculation process of T i (x i , y i ) is similar in (14) and (15).

Step~(The 2D Point Set T 2 is Converted to Dimensionally
. . , n. T i is the i-th dimensionality reduction point. Wherein x i , y i respectively represent the coordinate values of the X -axis and the Y -axis in the coordinate system X O Y after a cer- The 2D point set T 2 is converted to the dimensionality reduction point set T 3 , that is, the 2D point T i (x i , y i ) is converted to the dimensionality reduction point T i (x i , y i ). Here, T i (x i , y i ) is defined as (17).
Wherein the coordinates of the rotation center point T c of the dimension reduction point set T 3 is defined as (18).
θ is the angle at which the 2D point set T 2 is rotated counterclockwise with a center in T c from the positive direction of X -axis to the positive direction of Y -axis. A new 2D Cartesian coordinate system X O Y is established by rotating θ in the coordinate axis X O Y , wherein the coordinate value in X O Y corresponds to T 3 . All dimensionality reduction results are output in the coordinate system X O Y .
The angle θ is determined by comparing the different coordinate values of M and P in the 2D coordinate system X O Y . In the initial 3D coordinate system XYZ, M (a 1 , b 1 , c 1 )  and P(a 3 , b 3 , c 3 ) are defined. We assume that the values of

III. EXPERIMENTAL RESULTS
There are two ways to evaluate the dimensionality reduction of 3D-handwritten characters. The first approach is to consider visualizations of the results of dimensionality reduction. The second approach is to calculate the recognition rate of the image after dimensionality reduction.  Therefore, the experimental process in this paper is divided into two parts: one for visual comparison and another for character recognition. In order to verify the visual processing effect of the proposed method, we compared it with the dimensionality reduction results of four other algorithms: PCA, KPCA, MDS, and Isomap, which are mainstream and effective algorithms.
In order to compare the effectiveness of dimensionality reduction methods fairly and objectively, we select three most  common and simple recognition algorithms: k-nearest neighbor (KNN) [17], Support Vector Machine (SVM) [18], [19], and Naïve Bayes [20], [21]. We compare the recognition rate produced by these algorithms for the reduced-dimensional images obtained by five previously described dimensionality reduction algorithms. In order to highlight and compare the effectiveness of the dimensionality reduction method, the three most common and simplest identification methods are selected.

A. COMPARISON OF VISUALIZATION RESULTS OF DIFFERENT DIMENSION REDUCTION METHODS
The experimental platform is Sublime Text 2, and the experimental environment is Windows 10 operating system.  The experimenter wrote 26 3D lowercase letters by the tip of the index finger with Leap Motion.
As shown in Figure 8, the red curve represents the character trajectory for the letters a-z. The acquired 3D-handwritten character points are adaptively bounded by an oriented bounding box. The dimensionality reduction algorithms based on PCA, KPCA, MDS, Isomap and the proposed method, respectively, obtain the following visualization results, as shown in Figure 8.
Through the experimental results, it can be clearly found that the 2D characters obtained by the PCA, KPCA, MDS and Isomap will randomly produce mirror and angle rotations.  Since the direction of the dimensionality reduction image is not fixed, this will lead to a lower recognition rate. On the contrary, the proposed method overcomes the problem of mirror and angle rotations in the reduced-dimensional image by converting the projection point set T 1 to a 2D point set T 2 , and converting the 2D point set T 2 to a dimensionality reduction point set T 3 , therefore the visualization looks better than in other methods.

B. COMPARISON OF THE RECOGNITION RATES OF DIFFERENT DIMENSION REDUCTION METHODS
In this paper, the 3D-handwritten character trajectory is processed in a lowercase letters database. The data from the lowercase database is collected from 10 experimenters. Each experimenter recorded 26 lowercase letters 40 times. There are 10,400 samples in the database, 10-fold cross validation is used for training and testing. Then we calculate the  recognition rate of every single 3D-handwritten character in the lowercase database, and the average recognition rate of all characters.
The recognition rate of a single character can be calculated as (19).
Among them, R is the recognition rate of a single character. n 1 is the number of single 3D-handwritten characters to be identified correctly. N 1 is the total number of single 3D-handwritten characters.
For all samples in the lowercase database, single 3D-handwritten character recognition rates for five kinds of dimensionality reduction methods under three recognition methods are shown in Tables 1. In Table 1, under the different recognition methods, the value with the highest recognition rate is shown in bold.
As can be seen from Table 1, under the KNN, SVM and NB identification method, the single 3D-handwritten character recognition rate of the dimensionality reduction image obtained by the proposed method is higher than those obtained by PCA, KPCA, MDS and Isomap. Because the proposed method solves the problem of mirror and angular rotation of 2D characters after dimensionality reduction, the recognition rate is improved. Therefore, we obtained the coincident results to the visualization experiment section.
In addition, as the latest method of recognize the characters from the 3D tracks directly, the recognition rate of the method in literature [1] is also compared with that of the proposed method. From Table 1, you can see that under the SVM and KNN identification method, the proposed dimensionality reduction method can obtain better and comparable result, compared with the method in [1]. It is also can be seen from Table 1 that, beside the method of dimensionality reduction, the recognition rate is also dependent on the identification method of 2D characters.
For all samples in the 3D-handwriting lowercase character database, the average recognition rate of all characters processed by different dimensionality reduction algorithms are shown in Figure 9.
As can be seen from Figure 9, the average recognition rates of the proposed method under the KNN, SVM and NB are 89.9%, 91.5% and 86.9%, respectively. The similar conclusions as in Table 1 and Figure 8 can be obtained, showing that the average recognition rate of the proposed algorithm is better than for other algorithms. Similarly, the highest recognition rate is obtained because the problem of random mirror and angle rotations of the reduced-dimensional image is solved.

IV. CONCLUSION
In this paper, an algorithm for dimensionality reduction of 3D-handwritten characters based on an oriented bounding box was proposed. By converting the projection point set T 1 to a 2D point set T 2 , and converting the 2D point set T 2 to a dimensionality reduction point set T 3 , we solve the problem of random mirroring and angular rotation of traditional dimensionally reduced images. The experimental results indicate that the proposed algorithm can obtain 2D dimensionality reduction images in a fixed direction, and the proposed method improves the recognition rate by 32.6%, 31.4% and 32.4% under the three identification methods of SVM, KNN and NB.