Distinguishing Oracle Variants Based on the Isomorphism and Symmetry Invariances of Oracle-Bone Inscriptions

Oracle bone inscriptions are the earliest mature writing system discovered in China. Oracle is usually published in the form of glyphs and pictures, and the related oracle font database environment is closed and limited. Due to the lack of standardization of ancient oracle bone inscriptions, the structure and glyph of the same characters are not unified, and a large number of different shapes and complicated characters coexist. Even ancient character devotees need to complete the basic recognition of oracle with the help of professional reference books and experts. In addition, in the field of traditional literature, distinguishing variant characters of oracle-bone inscriptions needs strong expert knowledge and lacks efficiency. The recognition process consists of two stages. According to the characteristics of oracle-bone inscriptions, such as left-right symmetry, up-down symmetry and positive and negative coexistence, the corresponding data enhancement methods are used to get sufficient training samples in the first stage. In the second stage, the recognition candidate set under each threshold condition in the first stage are used to select the recognition candidate set which is empty under the specific threshold condition, and introduce the prior knowledge to distinguish the oracle variants. In the first stage, computer related methods are used to identify oracle variants. In the second stage, on the basis of the first stage, a set of recognition results that cannot be determined roughly is selected, and a priori knowledge is introduced to integrate multi-domain methods to identify oracle variants. The method proposed in this paper has achieved great results in the recognition of oracle variant characters.


I. INTRODUCTION
Oracle, an ancient Chinese character, is the source of the development of Chinese characters and the symbol of Chinese civilization for thousands of years. It has lofty symbolic significance [1], and is often engraved or written on tortoise shells and animal bones. Oracle-bone inscriptions were unearthed in Anyang City, Henan Province, China. Most of them are records of divination by the royal family of Shang Dynasty, and mainly about weather, agriculture, sacrifice, hunting and musical instruments. Although oraclebone inscriptions have begun to have a character system, and the modern Chinese character system gradually evolved on The associate editor coordinating the review of this manuscript and approving it for publication was Zhenhua Guo . the basis of oracle-bone inscriptions, there are still big differences between the two types of character systems. Oraclebone inscriptions are the starting point of the development of Chinese characters, where social and natural knowledge were mixed [2]. Although they are systematic, the shape and structure of the characters are not fixed, and the writing style of ancient oracle-bone inscriptions is not standardized. The oracle-bone inscriptions are characterized by a large number of homographs, complexity and simplicity. The structure of oracle characters is not fixed, the forms of single and compound characters are changeable, and the writing style of oracle characters is highly arbitrary, which is a difficult problem to be solved in oracle character recognition.
The artistry of oracle-bone inscriptions has been paid much attention by ancient character devotees, but most of them are difficult to use and recreate, which mainly caused by the recognition problem of oracle-bone inscriptions. And the main reason why oracle-bone inscriptions are difficult to be recognized is that oracle-bone inscriptions have a large number of complex and different variants.
The variant characters of oracle-bone inscriptions refer to the different shapes of the same characters. In the recognition of oracle variant characters in traditional literature, oracle experts need to analyze the changes of the basic structure of the single character when they recognize the single character. When distinguishing the oracle characters, oracle experts need to analyze the number and location of the basic components of oracle characters.
In the field of computer, the structure of oracle is the natural structure of oracle. The natural structure of oracle-bone inscriptions refers to the distribution of structural elements in spatial layout. Some oracle-bone inscriptions retain some characteristics of ancient architecture and ancient utensils, and the shapes and drawings presented are quite consistent with those of ancient architecture and ancient utensils. Although the standardization of oracle-bone inscriptions is far less than that of modern Chinese character system, oraclebone inscriptions are hieroglyphs. As a language communication symbol, they still need to write, interpret, transmit or record information, so as to avoid the situation of unrestricted writing and lay a foundation for the subsequent development of the character system. For the analysis of the natural structure of oracle-bone inscriptions, the commonly used research method is to classify the oracle-bone inscriptions according to the single or combined characters, and then analyze the combined characters from different configuration angles.
In the system of oracle-bone inscriptions, the proportion of monosyllabic characters is much higher than that of modern Chinese characters.
The study of the natural structure of the compound characters is about the spatial description of the characters. It needs to consider the change of the construction unit and its position, and finally analyze the overall layout of the characters formed by the construction unit. On the study of the combination types of oracle-bone inscriptions, the combination types of oracle-bone inscriptions can be divided into 7 groups and 14 types, namely, left-right separation and up-down superposition, vertical insertion and horizontal insertion, left-down filling and right-down filling, left-up filling and right-up filling, upper embedding and lower embedding, left embedding and right embedding, middle embedding and middle crossing [3]. Among them, the left-right structure and the upper and lower structure account for the most, while the left-right structure accounts for the most in the modern Chinese character system. The rich types of natural structure fully reflect the pictorial characteristics of oracle-bone inscriptions.
In the field of computer, oracle is not only a character but also a picture. In this paper, convolutional neural network is used to train the model, so as to recognize oracle variants. Convolutional neural network has been widely used in computer vision, information retrieval, natural language processing and other fields. Convolutional neural network takes the original image data as the input data, completes the feature learning from a large number of training data, and automatically learns the representation of features, making the information contained in its data easier to be extracted and analyzed [4]. Convolutional neural network has the characteristics of local connection, weight sharing and pooling operation. Through these characteristics, the number of parameters required by convolution neural network can be reduced, so as to reduce the complexity of convolution neural network, effectively reduce the risk of model over-fitting, and a certain degree of image zoom, translation, deformation and other operations can be accepted.
Convolution neural network is usually a multilayer supervised learning neural network, in which convolution layer and pooling layer are the key to feature extraction. The convolution neural network model uses gradient descent method and its variants to minimize the loss function, updates the weight parameters of the convolution neural network through back propagation, and improves the learning accuracy of the convolution neural network through multi round iterative training. Convolution neural network is a multilayer neural network composed of convolution layer and sampling layer, in which convolution layer is used for feature extraction and sampling layer is used for feature processing. The high-level full connection layer corresponds to the mapping between the classifier implementation of the multi-layer perception and the output target.
The training of neural network is mainly based on highorder matrix parallel operation. Nowadays, with the rapid development of computer hardware, high-performance and high-quality GPU provides strong parallel computing ability, and large-scale neural network training provides strong support.
Although oracle-bone inscriptions as characters have certain standardization, they still have the characteristics of graphics and characters. The writing style of oracle-bone inscriptions is relatively random, there are many kinds of oracle-bone inscriptions, and there are many configuration variations, which makes it difficult for computer to distinguish oracle-bone inscriptions. In the strict sense, there are 1032 groups of variant characters in oracle-bone inscriptions, with a total number of 3085, accounting for 49.50% of the total number of oracle-bone inscriptions and 51.91% of the total number of oracle-bone inscriptions [3]. Oracle-bone inscriptions are a systematic system of characters, and the language model is made up of corpora. Oracle's corpus is in a limited and closed environment, and the updating of oracle's corpus is very slow, and there are a small number of identified errors or uncertain oracle. In the field of computer, the recognition of oracle variant characters cannot use the modern Chinese character system recognition method. It needs to carry out targeted recognition according to the specific characteristics of oracle. When the original image data of oracle is input into convolutional neural network for training, the characteristics of oracle itself need to be considered. VOLUME 8, 2020 In addition to the characteristics of homograph, complexity, combination and pictograph, oracle-bone inscriptions also have the characteristics of positive and negative coexistence, left and right symmetry, up and down symmetry, rotation and change of meaning. The oracle variant characters have a lot of noise for traditional modern character recognition. The input data of neural network needs label information, but the cost of building a large-scale oracle image database is high, and manual annotation needs expert guidance and time-consuming. Through computer and other means, we can distinguish oracle variants, and assist experts to complete the recognition and confirmation of oracle variants.
Our main contributions in this paper are as follows: First, a systematic model is well defined to distinguish oracle variants. Second, a two-stage method is proposed, integrating multi-domain knowledge and providing an extensible framework. Third, the isomorphism and symmetry invariance of oracle-bone inscriptions are fully considered in both stages.
In this paper, a recognition of oracle variants method is proposed, based on the isomorphism and symmetry invariances of oracle-bone inscriptions. The recognition process can be divided into two stages. In the first stage, this paper uses VGG16 network based on transfer learning and finetuning method, and generates a large number of pictures that conform to the multiple characteristics of oracle characters by using the method of enhanced pictures, so as to expand the training samples, improve the generalization ability of the model and the recognition ability of oracle variants. In the first stage, we can get the recognition candidate set under different threshold conditions. In the second stage, we select the oracle font to be tested under the specific threshold conditions as the empty set, and introduce the prior knowledge and other information to identify the oracle variant. Through the two-stage method, combining the traditional literature field and the computer field, we can identify the oracle variant characters. In this paper, the accuracy and recall of the first k (k = 1, 2, 3) classification results are compared with the traditional model in accuracy, stability and time efficiency, which provides a reference for the subsequent comparison of this algorithm and its derivative application in related fields.
This article proceeds as follows. Section II reviews related work on recognition of oracle-bone inscriptions. In section III, the related characteristics and concepts involved in this paper are formally defined and standardized. Section IV introduces the specific two-stage method recognition method of oracle variant characters based on the isomorphism and symmetry invariances of oracle-bone inscriptions. Section V is the experimental setup and results analysis part. It analyzes the recognition accuracy of the first k classification results and the change of recall rate on the recognition candidate set under different thresholds, and compares with the traditional model. Finally, section VI of this paper gives the main conclusions and possible research directions in the future.

II. RELATED WORK
The research scope of oracle-bone inscriptions is extremely extensive. In the field of computer, oracle-bone inscriptions are glyph drawings and the rudiment of modern Chinese character system. Therefore, Zhou [5] and Li [6] put forward some recognition methods of oracle-bone inscriptions based on graph theory. They abstracted oracle as an undirected graph composed only of points and lines, and extract its topological features. The endpoint, cross point, block and hole of oracle glyph are regarded as the first level features and encoded. Among them, the end point refers to the vertex with degree equal to 1, the cross point refers to the vertex with degree greater than 2, the block refers to the connected branch in the graph, and the hole refers to the closed area composed of edges in the graph. Since the topological features are not deformed by displacement, extension and rotation, only one level features can identify the complex oracle-bone inscriptions with high stability, but for the relatively simple single character, the coding repetition rate is high, and the oraclebone inscriptions cannot be correctly identified. Therefore, it is necessary to add secondary features on the basis of primary features. Since the information of endpoint itself has not been fully considered, Zhou [5] took the stroke direction, stroke curvature and stroke bending times of oracle shaped drawings as secondary features and encodes them. Among them, the oracle stroke direction is defined as starting from a certain end point a of the oracle character, tracing along the relevant path until it meets a fork point or another end point, which is recorded as P, then the definition line PA is the generalized stroke [5]. Li [6] introduced the information of the adjacent points of the end points, and takes the cut set of the adjacent points, the adjacent subgraphs and the adjacent points as the second level feature, and then introduces the direction feature of the end points as the third level feature. Through the continuous recognition of multi-level feature coding, the accuracy of oracle recognition is improved.
However, due to the time-consuming and labor-consuming way of multi-level feature coding, ordinary users cannot master the complex coding rules, and only experts can master them skillfully, which makes it very difficult to distinguish oracle-bone inscriptions with the help of graph theory. Gu [7] analyzed the fractal nature of oracle-bone inscriptions, calculates the fractal dimension of oracle-bone inscriptions and matches it with the characteristic database of oracle-bone inscriptions, so as to identify oracle-bone inscriptions.
Fractal geometry is a kind of geometry with irregular geometry as its research object. The research object of traditional geometry is integer dimension, while the research object of fractal geometry is non-negative real dimension. Fractal is the study of infinitely complex geometry with self-similar structure. Gu [7] proposed the method of fractal geometry to identify oracle-bone inscriptions. Because of the large number of variants, the fractal characteristics of oraclebone inscriptions are quite different. Put the oracle shape picture into the plane, get the quadrant information, calculate the fractal dimension values of four quadrants respectively, and then get the arithmetic mean value of fractal dimension. In this case, a vector is used to represent the characteristics of the corresponding oracle shaped picture, and the fractal dimension eigenvector is used to describe an oracle shaped picture. Finally, the recognition results are obtained by matching the oracle to be recognized with the fractal feature database. In distinguishing the oracle-bone inscriptions, the method of analytic geometry needs to build fractal feature database for all known oracle-bone inscriptions, which is inefficient.
Liu [8] put forward the oracle character recognition method based on SVM, using machine learning related algorithm to recognize oracle character. However, due to the coexistence of complex and simplified oracle characters, the recognition accuracy still has room to increase, so it is necessary to further improve the recognition method of oracle-bone inscriptions. Besides, in Chapter 11 of [9], an unsupervised method was used to distinguish whether the oracle handwriting had been engraved with the right hand or left hand, and some interesting results were found.
In the field of traditional literature, only oracle experts can be competent for the recognition of oracle characters. The confirmation of oracle characters needs to be studied in many aspects. The core of the study of oracle characters is to determine the side and components of oracle characters. In a broad sense, the study of parataxis and component is to determine the ''radical'' in oracle characters. In the late Qing Dynasty, the study of paratactic forms of ancient Chinese characters was officially recognized. Later, the method of radical analysis continued to develop, and evolved into the study of today's oracle structure, grapheme, morpheme and so on. The characters of oracle-bone inscriptions are of the same shape, the side position and writing direction are not standardized, and the complexity and simplicity are different. Oracle experts need to determine the components in the oracle glyph first, and then confirm the glyph. In order to judge whether it is a homonym, we need to consider word examples and usage, and observe the language environment of the glyph. The traditional accurate recognition method of oraclebone inscriptions costs a lot of human, material and financial resources.
The recognition of oracle-bone inscriptions in the field of traditional literature needs a lot of professional knowledge. This recognition process is time-consuming and laborconsuming, but with high recognition accuracy. However, oracle recognition based on computer field can simultaneously generate a large amount of oracle image data, which has high recognition efficiency but less accuracy than that of traditional expert recognition. Based on the advantages of traditional literature and computer recognition, this paper proposes a two-stage method for the recognition of oracle variants.
In this paper, the two-stage oracle variants recognition method has the following characteristics: First of all, this paper combines the methods of traditional literature and computer pattern recognition, and then puts forward a two-stage oracle variant character recognition method, which can distinguish the oracle variant characters from different angles, and provides a more abundant reference for the second stage of recognition. Secondly, this paper uses convolutional neural network based on transfer learning to leverage the powerful feature extraction capabilities of the pre-trained model to improve the accuracy of variant word recognition and reduce model training time.
Finally, according to the selection of specific thresholds, the candidate sets with different precision can be selected. In the second stage, we can use different prediction and classification results of candidate sets under different thresholds in the first stage to assist in the recognition of oracle font to be tested in the second stage.

Definition 1 (Component):
It refers to the basic component of oracle characters, the construction unit of oracle characters, and the construction elements that directly or indirectly affect the reference construction characters. It is a split part of oracle characters and has the meaning of constructing the shape of oracle characters. Definition 5 (Rotational Semantics): It refers to the oracle glyph drawings or components after a specific angle of rotation no longer have the expression meaning of the previous text or components, so as to represent new text or new components.
Generally speaking, when the rotation angle of a text or component is more than 15 degrees, it is considered that the meaning of the text or component changes. The common change angles are 90 degrees and 180 degrees.
Definition 6 (Isomorphism): It refers to the character of the same oracle character with many big differences.
Definition 7 (Coexistence of Complexity and Simplicity): It refers to the characteristics of complexity and simplicity of writing in the same oracle glyph.
For example, for the same oracle character, it generally consists of three components, and each component can be written in several ways. Finally, the same oracle character has the characteristics of complexity and simplicity based on the writing of components.

Definition 8 (Collection of Original Oracle Glyph Drawings, I):
It refers to the collection of original oracle glyph drawings without morphological transformation.
Definition 9 (Enhanced Oracle Glyph Picture Set, EPS): It refers to a new picture set of elements in I generated by the method of targeted data enhancement based on set I according to the characteristics of oracle glyph picture.
Definition 10 (Recognition Candidate Set, RCS): It refers to that for each type of the oracle glyph pictures in the test set, a label whose classification result predicted by the training model is likely to be greater than a certain threshold value is selected as the recognition candidate set of the word.

IV. RECOGNITION METHOD OF ORACLE VARIANTS
This paper uses deep learning method to distinguish oracle variants. Oracle-bone inscriptions are mostly in the form of glyphs and pictures, with the characteristics of coexistence of complex and simple, homographs and so on. Heterographs account for more than half of all the glyphs of oracle-bone inscriptions, and the vast majority of oracle-bone inscriptions have very few samples, so the amount of data used to train neural networks for deep learning is far from enough. In order to meet these characteristics of oracle, this paper uses the method of data enhancement to expand the dataset to obtain training samples.
In this paper, the recognition method of oracle variants is mainly divided into two stages. In the first stage, we use data enhancement method to generate EPS for I in datasets and balance the number of each type of the glyphs to get training samples that meet the needs of convolutional neural network, where I refers to the original pictures of oraclebone inscriptions and EPS refers to the augmented pictures of oracle-bone inscriptions.
In order to improve the training efficiency of the traditional convolutional neural network and improve the prediction performance of the final model, this paper uses the VGG16 network based on transfer learning, and carries out multi label classification. Transfer learning is to transfer the network parameters pre-trained on large datasets to the new network, and freeze the relevant parameters to make it unable to continue training and updating, so that the new model has strong feature extraction ability on the pre training model, and can greatly reduce the training time. Transfer learning and corresponding fine-tuning methods can improve the recognition rate of the model. Through training, the corresponding model is obtained, and the recognition results are predicted on the independent test set, and the RCS of the oracle to be tested is obtained. Calculate the recognition accuracy of the first k (k = 1, 2, 3) results in RCS, and put the to-be-tested oracle glyphs whose RCS is an empty set under a certain threshold β into the second stage. In the second stage, the RCS of the oracle character to be tested under other threshold conditions in the first stage is introduced as the auxiliary recognition basis, and the prior knowledge such as single character, compound character, side information, description attribute, graph theory is introduced to help identify the oracle character to be tested whose RCS is an empty set, so as to obtain the recognition results of the oracle character to be tested. Prior knowledge is usually hard to obtain [10]. In this field, prior knowledge involves sacrificial rites, belief, geography, country, disease, childbirth, war, army, animal husbandry, hunting, celestial phenomena and agriculture.

A. CONVOLUTION NEURAL NETWORK
The input of the convolution neural network is the oracle glyph. The input image contains three channels R, G and B [11]. After the input image passes through several convolution layers and sampling layers, the mapping between the input image and the output object is completed in the full connection layer. The simple structure of convolutional neural network is shown in Figure 1.
Low level convolution layer can extract low level image features. Features are extracted and compressed continuously, and finally higher level image features can be obtained in higher level convolution layer. Low level convolution layer can extract low level image features. Features are extracted and compressed continuously, and finally higher level image features can be obtained in higher level convolution layer. The feature map is a component of convolution layer, and all neurons in each feature map share the same parameter, which is obtained by convolution operation of convolution kernel. The convolution kernel is used as the weight to multiply the pixel value of the corresponding block of the input image of the front layer, and finally the output pixel is obtained by the activation function. The n-th feature map matrix x n m of the m-th layer can be expressed as where g(·) is the activation function of neurons, N m is the combination of input characteristic graphs, ⊗ represents convolution operation, k n−1 rm is the convolution kernel matrix, and b r m is the bias matrix. The activation functions commonly used in neural networks include sigmoid function [12], ReLU function [13], tanh function [14], etc.
The sampling layer is also called the pooling layer. The convolution layer is used to extract features while the sampling layer is used to reduce the number of parameters, so as to retain effective information as much as possible while reducing the number of parameters. The convolutional neural network is often connected to one or several fully connected layers after passing through several overlapping convolution layers and sampling layers. When the former layer of the full connection layer is the convolution layer, the full connection layer can be regarded as the global convolution whose convolution kernel is h × u, where h is the width of the convolution result of the previous layer and u is the height of the convolution result of the previous layer. When the current layer is a fully connected layer, the fully connected layer can be regarded as a global convolution whose convolution kernel is 1 × 1.
The output layer of convolutional neural network is usually the logic regression layer. If softmax regression is used to convolute the output layer of neural network [15], the output layer is called softmax layer. The output layer of convolutional neural network represents the probability that the input oracle image belongs to a certain class i, where, ω is the weight parameter of the previous layer of the output layer, b is the offset parameter, and M is the total number of categories of the input image.

B. DATA ENHANCEMENT
In the deep learning, the problem of the model trained by convolutional neural network is the model over-fitting. The most convenient and effective way to avoid model over-fitting is to increase the number of training samples. The oracle glyph picture database environment is relatively closed and limited. It needs expert guidance and a lot of human, financial and material resources to collect the oracle image, so it is unlikely to expand the oracle image database in a short period of time, and the traditional method of manually adding annotation samples is not feasible. However, the convolutional neural network needs enough training samples, otherwise it cannot learn the required features from the training data. In order to increase the number of training samples and avoid manual description of oracle glyphs, this paper uses the method of data enhancement [16] to automatically generate a large number of related oracle glyphs. For any oracle glyph picture in I , because of the coexistence of positivity and negation, left and right symmetry, upper and lower symmetry, rotation and change of meaning, a large number of images that conform to the character of oracle-bone drawing can be generated by image transformation. The operation of image transformation in this paper includes: zoom, rotation, vertical deformation, horizontal mirror image and crosscutting transformation, (1) Zoom. Enlarge or reduce the oracle glyph in a certain range.
(2) Rotate. Rotate the oracle glyph in a small angle clockwise or anticlockwise direction.
(3) Tilt. Deform the oracle-shaped picture in four directions: up, down, left, and right.
(4) Flip. Turn the picture of oracle-bone inscriptions left and right.
(5) Shear. Incline the oracle glyph to a certain direction. Due to the different frequency of the common oracle glyphs and the different number of variants in each type of the oracle, we need to consider the number of training samples in each type of the training sets. According to each type of the oracle inscription I i in I and the number of variants, a certain number of ε i is used to generate enhancement pictures, thereby ensuring that the number of samples of each type in the training sample is relatively balanced.
where M I refers to the total number of oracle glyphs, M i refers to the total number of variants contained in type i oracle glyphs in I , and M N refers to the total number of training samples.
For any element i in I , where I refers to the original pictures of oracle-bone inscriptions, the image transformation operation described above will be performed according to a certain probability. The operation of image transformation is independent of each other, and the deformation degree of image transformation is within a certain range, which is determined by the deformation parameters. Data enhancement method expands the number of training samples and improves the generalization ability of the model. The operation of image transformation needs to be adjusted according to the coexistence of positive and negative, left and right symmetry, up and down symmetry and rotation variation of oracle glyph picture. Oracle font system was formed earlier, and the writing of oracle lacked planning, which resulted in different font sizes and varying writing angles. Therefore, two operations are considered in image transformation: zoom and rotation. In addition, the phenomenon of homographs is common in oracle bone inscriptions, and there are often positive and negative coexistence between variant characters, or the shape itself is left and right symmetrical. About 52.26% of the inscriptions with M i ≥ 2 have symmetry or coexistence of positivity and negation and about 42.56% of the inscriptions with M i ≥ 3 have symmetry or coexistence of positive and negative. However, the case of vertical symmetry or variants with vertical symmetry is rare, so in the actual image transformation operation, the horizontal image is mainly considered. If the upper and lower symmetry is considered too much, the characteristics of the real oracle glyph cannot be generated, so the characteristics of the real oracle glyph cannot be extracted. The glyph characteristics of some oracle are shown in Table 1.
Through the data enhancement method, the original smallscale and closed limited oracle glyph picture dataset is expanded to generate the number of training samples in line with the convolution neural network, which improves the robustness and generalization ability of the model.

C. VGG16 NETWORK BASED ON TRANSFER LEARNING
VGGNet [17] is a deep convolution neural network proposed by the visual geometry group of Oxford University. The VGG16 network consists of five convolution blocks, two fully connected feature layers and one fully connected classification layer. Each convolution block is composed of several convolution layers and a pooling layer, with a total of 13 convolution layers, 3 full connection layers and 5 pooling layers. In addition, in the same convolution block, the number of channels in the convolution layer is equal.
In traditional pattern recognition [18], such as handwritten numeral recognition [19], modern Chinese character recognition [20], the difficulty of obtaining training samples is low, and the scale of training set is huge. Compared with the traditional pattern recognition field, oracle font image database contains less data, and it is difficult to obtain a large number of training samples. In this paper, we expand the training samples by the method of data enhancement, and on this basis, we also use the transfer learning method training model. Transfer learning is an important part of modern computer vision [21]. Transfer learning is a model method that transfers the parameters of other trained models to the new model to continue training. Transfer learning accelerates the training efficiency of the new model by means of parameter transfer, thus avoiding the situation of training the model from scratch and providing strong feature extraction ability. In this paper, VGG16 network model which has been pre-trained in ImageNet dataset is used for parameter transfer learning.
Among them, the parameters in the convolutional layer can be expressed in the form of ''conv < K >-< C >'', K represents the size of convolution kernel, and C represents the number of channels. The parameters in the fully-connected layer can be expressed in the form of ''conv-< N >'', and N represents the number of neurons. There are two common fine-tuning methods [22], one is to freeze all convolution blocks of the pre-trained model, discard the full connection layer and output layer of the training model, redefine the new full connection layer and output layer, and add the pre-trained model. The other method is to freeze part of the convolution layer of the pre-trained model, usually freeze all convolution blocks except the last convolution block, discard the full connection layer and output layer of the training model, redefine the new full connection layer and output layer, and add the pre-trained model. Method one is usually suitable for pattern recognition with small amount of training sample data but high similarity of data. Method two is usually suitable for pattern recognition with small amount of training sample data and low similarity of data.
In the data enhancement stage, a random image is selected from each type of the oracle glyphs in I as the test set I test , and the oracle glyphs in I test are screened out in I . A large number of training samples can be obtained from the rest of the original oracle glyph drawing sets through image transformation operations such as zooming, rotating, tilting, flipping and shearing. I and the enhanced picture dataset constitute the training sample I train . Introduce the pre-trained model VGG16 for transfer learning, redesign the full connection layer and output layer of the model, freeze the convolution blocks 1 to 4, only train the convolution block 5 and the full connection layer, use the strong feature extraction ability of the pre-trained model, fine tune the model parameters to complete the model training.
In this paper, after convolution block 5, we first add the global average pooling layer (GAP) layer [23], and then access the full connection layer and sigmoid classifier of 1024 neurons. Sigmoid is used as the activation function and binary cross entropy [24] is used as the loss function of classification. Each output tag is regarded as an independent Bernoulli distribution. Sigmoid classifier is suitable for multi label classification. For multi label classification, a sample's label can contain multiple categories. This paper hopes that the trained model cannot only distinguish oracle variants, but also try to mine the correlation between different oracle characters. In order to prevent over-fitting of the model, dropout layer is added between the global level pooling layer and the full connection layer [25]. In the training stage, every input element of dropout layer will be preserved with probability p, whose value range is (0,1], so as to reduce the over dependence between neurons in the same layer and effectively reduce the phenomenon of over-fitting. the result of multi label classification predicted by RCS = Model if tp, and the probability of classification is greater than t After traversing the test set and according to the multi tag results predicted by the model, we can get the RCS of each type of the oracle glyphs that meets a certain threshold t, and select the most probable RCS as the prediction result of the oracle font. RCS may be an empty set.

Algorithm 2 Convolution Neural Network Training and
Convolution neural network training and model prediction based on transfer learning belong to the first stage of oracle variant character recognition. In this stage, each type of the variant characters is obtained from the original oracle glyph picture set as a test set, which is used to verify the recognition ability of the trained model. For the prediction results of the model, RCS is obtained by filtering under the condition of threshold t. By comparing the predicted tag set of RCS with the real tag set, the accuracy of the model is calculated.

D. PRIOR KNOWLEDGE MATCHING
In the first stage, the RCS and its probability value of the oracle glyph to be measured can be obtained by convolution neural network model under certain threshold t. The threshold t of RCS is equivalent to the screening index of the prediction results of convolutional neural network model, and the prediction results below this threshold will not be selected into the RCS of the oracle characters to be distinguished. In the first stage, a specific threshold β can be determined. According to the needs of specific recognition accuracy, it is considered that the first k results predicted by the model under the condition of the threshold β are all credible, which is recorded as top k (k = 1, 2, 3). However, if the probability of recognizing variants given by the model is lower than β, the recognition results will not be put into the RCS of the current oracle glyph to be tested. At this time, part of the RCS of the oracle glyph to be tested is an empty set. Due to the existence of a large number of variants on different datasets, and the large differences between the variants of some single oracle-bone characters, there is still a small probability that the model cannot correctly distinguish the variants of oracle characters. At this time, we need to introduce related prior knowledge for further variant character recognition of oraclebone characters whose RCS is an empty set, and define this stage as the second stage.
In the second stage, the prior knowledge matching mainly uses the knowledge of parataxis, oracle characters and graph theory. If the RCS of the oracle characters to be tested is an empty set, it is considered that under the condition of the specific threshold β, the model cannot correctly distinguish the oracle variant characters. At this time, the text to be tested with RCS as the empty set were screened out, starting from the oracle radical, supplemented by the oracle characters description attribute information, graph theory knowledge and RCS information under other threshold α conditions(α < β), so as to identify the oracle variants. Generally, the value of α is small. For the convenience of expression, RCS under certain threshold β is recorded as RCS -β, and RCS under threshold α is recorded as RCS -α.
The core idea of this algorithm is to regard the RCS recognition result under the specific threshold β as the correct result, and the value of β is often higher. For the RCS with empty set under the condition of high threshold, the results need to be further identified and related prior knowledge introduced. In the second stage, some prior knowledge involves the RCS of empty set under β condition, Match the current recognition of oracle font and the graph theory information in the image database, determine the recognition results according to the graph theory matching results and add them to V and the oracle characters to be further identified are selected. Under the condition of lower than the threshold of β, the RCS of oracle is selected as the prior knowledge of the second stage.
In the second stage, the idea of distinguishing oracle characters in the field of traditional literature was used, and the prediction results of entering RCS were filtered according to the specific threshold β, and ACCS o was obtained by using RCS-related prediction information of other threshold α. Then judge whether the current oracle is a single character or a compound character, and screen the glyph to be selected in ACCS o from the side or single character matching information. Then judge whether ACCS o is an empty set. If the ACCS o is not an empty set, judge whether the description attributes of the selected characters and the matching information of graph theory are consistent with the oracle characters to be tested to give the recognition results of variant characters. If ACCS o is an empty set, we can directly compare the graph theory information in oracle database according to the graph theory matching information to get the result of variant character recognition.
The second stage uses the results of the first stage to reduce the number of potential recognition categories of oracle variants to be distinguished with accuracy below a certain threshold β accuracy rate, and improves the efficiency of traditional oracle variant recognition. At this stage, according to the different values of β, a variety of oracle-shaped characters with an empty set of RCS can be filtered. And according to different values of α, the RCS of the word given by the model under different threshold conditions can be obtained as reference information, which can further help identify oracle variants.

V. EXPERIMENTS AND ANALYSIS A. DATASETS
The dataset of this paper includes two parts: the original oracle image and the enhanced oracle image. Let M i denote the total number of variants contained in type i oracle in I . According to the standard of the total number of variants contained in oracle font, it can be divided into three kinds of datasets, i.e., Oracle-2, Oracle-3 and Oralce-4. Table 2 shows the relevant information of the datasets used in the experiments, including the number of categories, the number of training samples, the number of test set samples and image size.
In I train , each type of the oracle-bone inscriptions corresponds to a label. All images of I train need to be normalized, and the pixel value of the image needs to be normalized to [0,1], which is conducive to the training of the model.
Many inscriptions in oracle have a large number of variants, some of which are similar in morphology, and some of which are quite different, as shown in Table 3.  For example, the oracle-bone script corresponding to the Chinese .character '' '' in Table 3 has a similar left-right symmetrical oracle variant inscription, and the single oraclebone script corresponding to the Chinese character '' '' has a left-right symmetry.
In the M i ≥ 2 oracle forms, 52.26% of them have symmetry or coexistence of positive and negative. In the M i ≥ 3 Oracle characters, 42.56% of them have symmetry or coexistence of positive and negative characters. In the M i ≥4, Oracle characters, 39.18% of them have symmetry or coexistence of positive and negative characters. According to this feature in data enhancement, the horizontal mirror operation in image transformation takes different operation probabilities according to the symmetry of Oracle-2, Oracle-3 and Oracle-4 datasets. In this paper, we use the Augmentor library [26] in Python to generate enhanced pictures. Table 4 lists the image transformation operation, image transformation range and the probability of using the operation on the original oracle shaped picture.
Among them, the value of zoom represents the multiple of the original image size. The value of rotation indicates the degree of rotation, positive indicates clockwise rotation and negative indicates counterclockwise rotation. The value of vertical deformation indicates the degree of deformation of the image in the Augmentor library. The value of crosscutting transformation indicates the degree of deformation towards a certain direction in the Augmentor library. The enhanced image effect is shown in Table 5.

B. EVALUATION INDEX
In this paper, the training of convolutional neural network based on transfer learning is completed in the first stage, and the test set is completely independent of the training set. Sigmoid activation function is used in the output layer of the model for multi label classification. In the first stage, we can get the recognition result of variant characters of each oracle character. According to the size of threshold t, the multi label results predicted by the model are selected into the RCS, which may be an empty set. In this paper, the prediction results (k = 1, 2, 3) of the first k (top k) in the RCS of oracle glyphs with different threshold t are calculated to calculate the accuracy of variant character recognition of the model. At the same time, the recall of variant character recognition in the RCS of oracle under different threshold t is calculated.

C. EXPERIMENTAL RESULTS
In this paper, we compare the ability of recognizing oracle variants between traditional VGG network and VGG network based on transfer learning on Oracle-2, Oracle-3 and Oracle-4 datasets. The training of convolutional neural network adopts Adaptive Moment Estimation (Adam) optimization algorithm, and the learning rate of traditional Stochastic Gradient Descent (SGD) method remains unchanged in neural network training. Adam designs different adaptive learning rates for different parameters of the network by calculating the firstorder and second-order moment estimates of the gradient. The super parameters of neural network include initial learning rate θ, training rounds Epochs, regularization parameter λ, dropout layer parameter η and one-time training sample number BatchSize. The specific parameters and parameter values are shown in Table 6.
The convolutional neural network is fully trained in the first stage and predicts each picture in the test set to determine whether the prediction results meet the requirements of threshold t, so as to obtain the RCS of each type of the oracle glyph pictures to be tested. In the first stage, on the Oracle-2, Oracle-3 and Oracle-4 datasets of traditional VGG network, the results of top k accuracy, recall and F1 value of oracle variant recognition for different threshold t are shown in Table 7.
Based on Table 7, it can be found that under the same threshold t and k, whether in terms of the accuracy or recall of variant character recognition top k (k = 1, 2, 3), the overall recognition effect of Oracle-2 on traditional VGG network is better than that on the Oracle-3 and Oracle-4 datasets. In the same dataset, the accuracy and recall decrease with the increase of RCS threshold t, and the accuracy of variant recognition top k is greatly affected by k value in the range of t ∈ [0. 1,5]. In most cases, the top k accuracy rate is equal to the recall, which shows that the model has been able to accurately most of the oracle shapes, only when t is in a small condition, the recall will be higher than the VOLUME 8, 2020  top k accuracy rate. The results of top k accuracy, recall and F1 value of oracle variant recognition on Oracle-2, Oracle-3 and Oracle-4 datasets of VGG network based on transfer learning in the first stage are shown in Table 8.
By comparing Table 7 and Table 8, it can be found that the top k accuracy on the Oracle-2, Oracle-3 and Oracle-4 decreases with the increase of threshold t whether it is a traditional VGG network or a VGG network based on transfer learning. On the Oracle-2, the oracle variant recognition ability of VGG network based on transfer learning is better than that of traditional VGG network. In terms of recall, the traditional VGG network is better than the VGG network based on transfer learning when the threshold value is small. However, with the increase of threshold, the recall of traditional VGG network decreases dramatically, while the VGG network based on transfer learning still maintains a high recall.
On the Oracle-3, the oracle variant recognition ability of VGG network based on transfer learning is better than that of traditional VGG network as a whole. Except when the threshold value is very small (t = 0.1%), the recall rate of VGG network based on transfer learning is slightly lower than that of traditional VGG network. When the threshold value is in other states, the recall of VGG network based on transfer learning is better than that of traditional VGG network, and with the increase of threshold value, the recall rate is stable and slowly decreases.
On the Oracle-4, the overall oracle variant recognition performance is quite similar to that on the Oracle-3. When the threshold is at a high value, the recall rate decreases faster than that on the Oracle-3 but slower than that on the Oracle-2.
By comparing the performance of the two kinds of convolutional neural networks on Oracle-2, Oracle-3 and Oracle-4, we can find that the traditional VGG network shows better recall when the threshold value is t = 0.1% on Oracle-2 and Oracle-3. The recall rate of VGG network based on transfer learning at t = 0.1% differs from that at t = 90% on the Oracle-2, Oracle-3 and Oracle-4, respectively by 6.89%, 9.03% and 15.12%. The recall rate of traditional VGG network at t = 0.1% differs from that at t = 90% on the Oracle-2, Oracle-3 and Oracle-4 dataset, respectively by 30.40%, 19.14% and 22.34%.Traditional VGG network has great changes in recall rate, while VGG network based on transfer learning has a more stable performance in recall rate.
In the top k accuracy of Oracle-2, the top k accuracy of traditional VGG network at t = 30% is equal to the recall of RCS, while the top k accuracy of VGG network based on transfer learning at t = 40% is equal to the recall of RCS. On the Oracle-3, the top k accuracy of traditional VGG network at t = 20% is equal to that of RCS, while the top k accuracy of VGG network based on transfer learning at t = 70% is equal to that of RCS. On the Oracle-4, the top k accuracy of traditional VGG network at t = 10% is equal to that of RCS, while the top k accuracy of VGG network based on transfer learning at t = 20% is equal to the recall of RCS.
This shows that on the Oracle-3, VGG network based on transfer learning has better recognition results than traditional VGG network, and can still contain correct recognition results under high threshold conditions. Oracle-2 has more classes of oracle variants than Oracle-3. In the new class of oracle (M i =2), Oracle-2 has more similar variants. After data enhancement, it is conducive to the training and recognition of convolution neural network. Both traditional VGG network and VGG network based on transfer learning perform better than Oracle-3 on Oracle-2.
Oracle-4 has less classes of oracle variants than Oracle-3. Oracle-4has part of the classes (M i = 3) removed compared to Oracle-3. Both traditional VGG network and VGG network based on transfer learning on Oracle-4 performs better than that on Oracle-2 but worse than that on Oracle-3 at a whole.     By comparing Figure2, Figure 3 and Figure 4, it can be found that at t = 5% on Oracle-2, the top k average accuracy and recall obtained by both the traditional VGG model and the VGG model based on transfer learning are very similar. After that, the curves of the top k average accuracy rate and the recall rate under the same model gradually coincide. On Oracle-3, when the traditional VGG model is still under t = 5%, its top k average accuracy rate is close to the recall rate. When the VGG network based on transfer learning is t = 20%, its average accuracy of top k is similar to the recall rate. On the Oracle-4, when the traditional VGG model is under t ≥10%,the curves of the top k average accuracy rate and the recall rate gradually coincide. And when the VGG network based on transfer learning is under t ≥5%,the curves of the top k average accuracy rate and the recall rate are parallel.
On the whole, when the threshold t is at a small value, all types of models have a better ability to discover the correlation between the oracle variants.
The data enhanced image transformation operation will affect the training samples of the model, thus affecting the performance of the oracle variant recognition of the final model.
On the Oracle-2, the average top k accuracy and average recall with the change of threshold value under different image transformation conditions are shown in Figure 5 and Figure 6, respectively.
Among them, the final model refers to the VGG16 model based on transfer learning, and its training samples are obtained by a total of 5 image transformation methods including zooming, rotating, tilting, flipping, and shearing. The other lines indicate the average top k accuracy and average recall of the corresponding model obtained by canceling one of the image transformation methods. When t ≥5%, the average top k accuracy rate is basically equal to the average recall, so the average top k accuracy rate curve and the average recall curve are only different in a small threshold stage.
On the Oracle-2 dataset, zooming and shearing can improve the recognition accuracy and recall rate of the model in all threshold range. The rotation transform can slightly VOLUME 8, 2020  reduce the recognition accuracy of the model in the range of medium threshold, but can improve the recognition accuracy of the model in the range of low threshold and high threshold. Tilting can improve the recognition accuracy of the model in the range of higher threshold and lower threshold.
Similarly, on the Oracle-3 dataset, the average top k accuracy rate and average recall with threshold changes under different image transformation conditions are shown in Figure 7 and Figure 8, respectively.
Among them, when t ≥ 20%, the average top k accuracy rate is basically equal to the average recall, so the average top k accuracy rate curve and the average recall curve have large differences only at the small threshold stage.
On the Oracle-3 dataset, only the scaling operation will reduce the model's recognition accuracy and recall within a higher threshold range, and the other deformation operations improve or maintain the recognition accuracy of the current model within the full threshold range.  Also, on the Oracle-4 dataset, the average top k accuracy rate and average recall with threshold changes under different image transformation conditions are shown in Figure 9 and Figure 10, respectively.
Among them, the average top k accuracy rate curve and the average recall curve are different in a small threshold stage(t ≤ 5%). Rotating and shearing can improve the recognition accuracy and recall rate of the model to a great degree. The overall trend of precision curve and recall curve on the Oracle-4is similar to that on the Oracle-3.
To show the structure of RCS clearly, partial results of RCS in the first stage at t = 0.1% on the Oracle-2 and Oracle-3 are shown in Table 9.
In Table 9, the traditional VGG network has shown a good ability to mine the correlation between oracle variants in the first stage of RCS, while the VGG network based on transfer learning often shows the characteristics of high probability recognition results and few candidate words in the RCS under the same oracle glyph. On the Oracle-2, the RCS of traditional  VGG network has more alternative glyphs, while on the Oracle-3, the number of alternative glyphs of traditional VGG network is slightly lower than that of Oracle-2. In the VGG network based on transfer learning, the total number of alternative recognition glyphs of RCS is slightly lower than that of the traditional VGG network, but it has a higher recognition probability.
The results compared with the methods introduced in the related work are shown in Table 10.
Since oracle-bone inscriptions do not share a universal dataset, this article only compares the results provided in the relevant papers. The two-stage method (Oracle-2) performs better than the existing method. The compared methods take the complete dataset (Oralce-2) as the experimental object and do not subdivide the dataset. Oracle-3 and Oracle-4 have the more complex oracle-bone inscriptions with more variant numbers. On Oracle-3 and Oracle-4 datasets, the accuracy will be slightly lower than that on Oracle-2 dataset. If the oracle-bone inscriptions with only two variants are removed, the two-stage method still performs well and is far superior to the SVM method. Two-stage method performs well on complex datasets (Oracle-3 and Oracle-4).
Generally speaking, in the first stage, according to the distribution of Oracle-2, Oracle-3 and Oracle-4 variants, training samples that meet the input requirements of convolutional neural network are generated by reinforcement learning. Through experiments, the VGG16 network based on transfer learning is better than the traditional VGG network in the top k recognition accuracy of oracle variants, and the stability of recognition probability is far higher than the stability of the traditional VGG network.
The main performance is that with the increase of threshold t, the top k recognition accuracy of VGG network based on transfer learning is higher than that of traditional VGG network under the same threshold condition.

VI. SUMMARY AND PROSPECT
Oracle is the ancient code that carries the genes of Chinese civilization, the source of Chinese character development, and an indispensable part of the five thousand years of Chinese civilization. It has been 120 years since it was discovered by Y.R. Wang in 1899, but the academic research of oracle-bone inscriptions still keeps a certain distance from the public, which seems to be ''supercilious''. At present, there is a huge demand for popular cultural consumption, among which there are people who like oracle. However, due to the difficulty of specialization, even the recognition of an ordinary oracle script requires professional reference books and materials to get the answer, let alone use and artistic recreation. In order to improve the efficiency of oracle-bone inscriptions recognition and expand the ''convenience'' and ''practicability'' of oracle-bone inscriptions, this paper proposes a two-stage method of oracle-bone inscriptions variant character recognition.
As a kind of early hieroglyph, the boundary between the characters and symbols of oracle-bone inscriptions is not clear, and the glyph drawing is still an important characteristic attribute. Due to the lack of standardization of ancient oracle writing style, many oracle characters have a large number of variant characters, and there may be great differences in the configuration angle of the same oracle characters. After summing up, the isomorphism and symmetry invariances of oracle-bone inscriptions are the most distinctive features.
In the first stage, according to the feature distribution of the original oracle glyph picture dataset, the corresponding training samples are expanded through reinforcement learning, and the required model is obtained through the VGG16 network based on transfer learning. By calculating the top k accuracy rate on the prediction set and the recall on the RCS, we can compare the performance of VGG16 network based on transfer learning with that of traditional VGG16 network in oracle variant recognition. With the help of computer pattern recognition and other methods, models under different thresholds can be obtained through this stage.
In the second stage, based on the traditional method, according to the threshold requirements of specific targets, oracle variant characters can be accurately identified with the help of RCS information under different thresholds of the previous stage, the information of single or compound characters, component information, side information, description attribute information and graph theory knowledge of oracle glyph. The second stage is based on the first stage of threshold screening, which saves a lot of time, manpower, material and financial resources compared with the traditional oracle variant recognition. According to the value of specific threshold value, any RCS information that conforms to the prediction accuracy of the model can be selected as a prior knowledge to carry out the second stage of oracle variant character recognition. In this stage, a framework is proposed to improve the accuracy of oracle-bone inscriptions recognition. The selection of different threshold values determines the accuracy of the model used. High threshold values can filter out some ambiguous recognition results and only retain the recognition results that the model considers the most reliable. The correlation between different oracle-bone inscriptions can be found under the condition of low threshold. Once the threshold is set, based on the results of the first stage, the introduction of additional knowledge will improve the accuracy of recognition. The more knowledge introduced, the more the process is like ''looking up a dictionary''. In this stage, changes can be made according to the user's needs.
In the first stage, the recognition method of oracle variant based on two-stage method uses the method of computer pattern recognition to train the model and predict the oracle characters to be tested. This stage embodies the ''convenience'' and ''practicality'' of oracle, so as to help amateur oracle enthusiasts to identify oracle glyphs with high probability. In the second stage, with the help of the recognition ideas in the field of traditional literature, a framework of oracle variant character recognition is proposed. The RCS obtained in the first stage is transformed into prior knowledge according to the threshold value, supplemented by information such as glyph, component, radical and graph theory, so as to accurately identify oracle glyph. The recognition framework proposed in this stage reflects the ''academic'' characteristics of oracle-bone inscriptions. When it is necessary to accurately identify oracle-bone inscriptions, the information obtained by computer method can be used as a prior knowledge to assist in the identification of oracle-bone inscriptions. The two-stage method of oracle variant character recognition proposed in this paper integrates the recognition methods in the field of computer and traditional literature, retains the ''academic'' characteristics of oracle, expands the ''convenience'' and ''practicability'' of oracle, and achieves relatively satisfactory results. There is still a way to improve the recognition of oracle variants. Through further detailed analysis of the characteristic distribution of oracle glyph dataset, more methods can be used to generate training samples in the selection of data enhancement parameters, which is more in line with the real situation. The related research of oracle-bone inscriptions may also involve the research of handwriting, single character and the configuration of compound characters.