A New Fast Local Laplacian Completed Local Ternary Count (FLL-CLTC) for Facial Image Classification

Face recognition is one of the most interesting areas of research areas because of its importance in authentication and security. Differentiating between different facial images is not easy because of the similarities in facial features. Human faces can also be covered obscured by eyeglasses, facial expressions and hairstyles can also be changed causing difficulty in finding similar faces. Thus, the need for powerful image features has become a critical issue in the face recognition systems. Many texture features have been used in these systems, including Local Binary Pattern (LBP), Local Ternary Pattern (LTP), Completed Local Binary Pattern (CLBP), Completed Local Binary Count (CLBC) and Completed Local Ternary Pattern (CLTP). In this paper, a new texture descriptor, namely, Completed Local Ternary Count (CLTC), is proposed by adding a threshold value for the CLBC to overcome its sensitivity to noise drawback. The CLTC is also enhanced by adding the Fast-Local Laplacian filter during the pre-processing stage to increase the discriminative property of the proposed descriptor. The proposed Fast-Local Laplacian CLTC (FLL-CLTC) texture descriptor is evaluated for face recognition task using five different face image datasets. The experimental results of the FLL-CLTC showed that the proposed FLL-CLTC outperformed the CLBP and CLTP texture descriptors in term of recognition accuracy. The FLL-CLTC achieved 99.1%, 86.93%, 93.21%, 84.92% and 99.15% with JAFFE, YALE, Georgia Tech, Caltech and ORL face image datasets, respectively.


I. INTRODUCTION
Face recognition is one of the most widely used systems in the field of security [1], [2]. An efficient face recognition system should be able to recognize similar faces even with the noise, wearing glasses, different facial expressions [3], different views, postures or illumination [4]. Figure 1 shows that different images of same person could look quite different because of various conditions. The first process for the face recognition system is the face detection. This step involves determining whether an image contains a face. If the image has a face, it will proceed to the next step, which is The associate editor coordinating the review of this manuscript and approving it for publication was Inês Domingues . face extraction, wherein some descriptors from of the face can be extracted. Different types of feature descriptors have been presented for use in the feature extraction process. These features showed a good performance as powerful features and can be used to differentiate between different facial images. Local Binary Pattern (LBP) is one of the most commonly known texture descriptors, which was initially proposed for texture image classification, but its use has since expanded to various kinds of image processing tasks [5]- [11]. From LBP texture descriptor, different types of texture descriptors have been proposed such as Local Ternary Pattern (LTP) [12], Completed Local Binary Pattern (CLBP) [13], Completed Local Binary Count (CLBC) [14] and Completed Local Ternary Pattern (CLTP) [15], [16]. The superiority of LBP is in its robustness against the monotonic dim level amendment, and its simplicity in terms of extraction and execution. However, it also has some drawbacks. The LBP is sensitive to noise and different patterns of LBP might be arranged into a similar class, which will cause a decrease in its discriminative property [17], [18].
Some of the texture descriptors that have been proposed, such as CLBP and CLBC, have inherited the LBP drawbacks especially its noise sensitivity. While the CLTP displays the most robustness to noise compared to other texture descriptors such as LBP, CLBP and CLBC [19], the CLTP descriptor length is longer than others, although it still performs better in different applications [17], [19], [20]. In this paper, we propose a new enhancement to the CLBC texture to overcome the noise sensitivity problem. A new texture descriptor, namely, Completed Local Ternary Count (CLTC) is proposed by adding a threshold to the CLBC during the extraction process. This threshold will be selected empirically and will depend on the nature of the selected database. The proposed descriptor is also integrated with the Fast-Local Laplacian filter to increase the discriminative property of the proposed texture descriptor. The Fast-Local Laplacian (FLL) is used because of its extensive image detail enhancements and tone mapping. The proposed Fast-Local Laplacian CLTC (FLL-CLTC) is evaluated for face recognition using different face image datasets. The paper is structured as follows. Sections II and III provide a brief description of the face recognition system and some of the related texture descriptors. In Section IV, the proposed FLL-CLTC is explained in detail. The experimental setup and results of the proposed texture descriptor for face recognition are presented in Section V. Section VI contains the conclusions of the paper.

II. FACE RECOGNITION SYSTEM
Programmed face recognition has become one of the main face analysis assignments in the field of security. Face recognition has a very complex structure and contrasts extremely from others in different aspects [21]. Thus, the improvement for face recognition system could be a difficult undertaking in image processing of computer vision. Feature extraction is an important part of the face recognition system. Feature extraction is the procedure in which certain highlights of enthusiasm inside a picture are recognized and represented for additional handling to recognize a face in an image. The resulting representation can be used as a contribution to various pattern recognition and classification techniques, which will then label, classify, or recognize the semantic substance of the image. The general face recognition process is shown in Figure 2.

III. RELATED WORK
In this section, a brief review of different types of texture descriptors, namely, LBP, LTP and CLBC, is given. A brief explanation of the Fast-Local Laplacian filter is also provided.

A. LOCAL BINARY PATTERN (LBP)
In 1996, Ojala et al. [6] proposed the LBP to depict the neighbourhood grey-level attributes of surface composition. LBP is one of the simplest texture descriptors widely used in various applications. The algorithm of LBP compares the 3 neighbourhoods of every pixel with the value of the centre pixel and produces the binary form followed by the decimal number as an outcome. LBP can distinguish the spatial structure of an image. However, it suffers from some drawbacks, namely sensitive to noise because of the use of the centre pixel as a threshold and sometimes different patterns of LBP possibly could be classified into the same class. Despite its drawbacks, many different standard texture descriptors have been proposed to overcome the limitations of LBP and instead enhances it. LBP calculation is shown mathematically in Equation 1.
where i p (p = 0, . . . , P − 1) refers to the grey values of neighbours pixels, i c refers to the grey value of the centre pixel, P is the number of neighbours and R refers to the radius of the texture pattern. The bilinear interpolation method is used to determine the exact values of the neighbours' pixels on the R radius that are not in the centre [7]. Figure 3 shows that the LBP extraction algorithm contains two main steps, namely the thresholding step and the encoding step. In the thresholding step, all neighbouring pixel values in each pattern are compared with the value of the central pixel of the pattern to convert the values to a binary value (0 or 1). This step enables the obtaining of information on local binary differences. In the encoding step, the binary numbers obtained from the thresholding step are encoded and converted into a decimal number to characterise a structural pattern.

B. LOCAL TERNARY PATTERN (LTP)
In 2010, the Local Ternary Pattern (LTP) was proposed by Tan and Triggs [12] to overcome the noise-sensitivity of the LBP and enhance its performance. In LTP, a user-specified threshold (t) is used to modify the original LBP to obtain three encoded values, namely, −1, 0 and 1 by comparing the differences between centre and neighbourhood pixels. Mathematically, the LTP can be presented in Equation 2 as follows.
where t is a user-specified threshold as shown in Figure 4.

C. COMPLETED LOCAL BINARY COUNT (CLBC)
The Local Binary Count (CLBC) descriptor was proposed in 2012 by Zhao et al. [14]. In the LBC, the encoding step is removed and only the number of value 1's in the binary neighbour group are counted. Figure 5 shows the comparison of the centre pixel value with its neighbours to obtain 1 or 0. The number of 1 values are the only ones counted. Hence, in Figure 5, the local binary count code of the centre pixel is 5, which reflects the number of 1 values. Mathematically, the CLBC can be presented in Equation 3 as follows.
where i c , i p , c and c I are defined in Equation 1. The LBC is extended to completed LBC (CLBC). The CLBC_S, CLBC_M and CLBC_C operators were combined into joint or hybrid distributions and used for rotation invariant texture classification. The CLBC_S, CLBC_M and CLBC_C can be described mathematically as follows.
where i c and i p are defined in Equation 1. c is the mean value of m p in the whole image and the c I is the average grey level of the whole image.

D. FAST-LOCAL LAPLACIAN FILTER
The Fast-local Laplacian (FLL) filter is an edge-aware filter based on the local Laplacian pyramid. The Local Laplacian filter was proposed in 2011 by [23], a researcher at Adobe, who investigated the Local Laplacian filter and determined that it provides a wide range of parameters while simultaneously producing high-quality visuals for detailed manipulation and tone mapping. The filter is also determined to produce strong and extensive detail enhancements compared to existing recent filters in terms of visual quality. However, the computational time of local Laplacian filter is slow, with only a minute per megapixel with a single thread, thereby requiring parallel implementation and an approximation scheme to reach interactive rates. Therefore, the FLL was introduced to control the trade-off between speed and accuracy [24]. The FLL is used for sharpening the images and removing noise. The filter is especially skilful at disclosing fine details in the image. Texture descriptors search for fine details of the texture information, making this filter suitable for the texture features [25]. The procedure of FLL can be isolated into three main steps. The output image is obtained when the output pyramid collapses in FLL. Firstly, this filter utilises the Gaussian pyramid to process the input image. Secondly, it integrates all the linear interpolations and calculates every coefficient output of the changed image's Laplacian pyramid. Thirdly, the output image is obtained by collapsing the output pyramid [26]. FLL approximates the calculation by discretising the intensity limits into various examples characterised by the number of intensity levels to accelerate processing. The number of intensity levels can be utilised to adjust the speed and quality. Figure 6 illustrates the effects of FLL on an example of a face image.

IV. PROPOSED FAST-LOCAL LAPLACIAN COMPLETED LOCAL TERNARY COUNT (FLL-CLTC)
In this paper, we are proposing a new texture descriptor, namely CLTC by improving the CLBC by adding a threshold to be more robust to noise. Furthermore, the proposed CLTC is integrated with the Fast-Local Laplacian filter (FLL) to increase its discriminative property. The Fast-Local Laplacian CLTC (FLL-CLTC) is evaluated in this paper for face recognition task using different benchmark face databases.
The general structure of the face recognition system using the FLL-CLTC stages is shown in Figure 7. After applying the FLL on the image, the CLTC is extracted from the image. Different texture pattern sizes can be used to extract the proposed CLTC. The pattern radius size ranges have many sets and the most used recognized sets are (1,8), (2,16) and (3,24). These radius sizes can turn out with an arrangement of different results as shown in Figure 8. These pattern sizes are selected in our experiments. In the example in Figure 9, a 3 × 3 texture pattern is used to represent each pixel in the image. The centre of each texture pattern i c is added and subtracted from the threshold (t) to generate two different patterns, namely, upper and lower patterns. The local differences between each new centre value and its neighbours will be used to calculate the upper and lower histograms. Thus, the local differences of each pattern are decomposed into two sign complementary components S upper p and S lower p as shown below.
s lower VOLUME 8, 2020 The third part is the FLL − CLTC center. The FLL − CLTC_C upper P,R and FLL − CLTC_C lower P,R are expressed mathematically as follows: The steps of the proposed FLL-CLTC extraction process are shown in Figure 11.

V. EXPERIMENTS AND DISCUSSIONS
This section presents a series of experiments to evaluate the proposed FLL-CLTC. Five benchmark datasets are used to evaluate the proposed FLL-CLTC, namely, JAFFE [28], YALE [29], Georgia Tech Face [30], Caltech Pedestrian Faces 1999 [31] and ORL [32]. In these experiments, the classification results of the proposed texture descriptor are compared with CLBP and CLTP as well as some previous works. For validation, each experiment is repeated 100 times with random training images and the final recognition accuracy is obtained by determining the average of the number of the repetitions.

A. DISSIMILARITY MEASURING FRAMEWORK
In this paper, the nearest neighbourhood is used as a classifier while the χ 2 statistics as metrics to calculate the dissimilarity between two histograms [14]. The χ 2 distance between two histograms H = h i and K = k i where (i = 1, 2, 3, . . . B) can be described mathematically as shown in Equation 19.  Figure 12.

C. YALE FACE DATABASE (YALE)
The Yale Face database is constructed by Yale University in 1997 and consists of 15 individuals in different conditions [29]. Each individual is captured in 11 images for the purpose of face recognition and facial expression recognition. The entire dataset consists of 165 images. Each image is captured from different positions, illumination brightness and face expressions. Some images from the YALE database are shown in Figure 13. The experimental results of the proposed FLL-CLTC, CLBP and CLTP with YALE dataset are shown in Table 2. The results showed that the proposed FLL-CLTC outperformed the CLBP and CLTP in (1,8), (2,16) and (3,24)  scale and illumination conditions. Some of the subjects also wore eyeglasses and had small-sized images with low resolution. Examples of Georgia Tech Face images are shown in Figure 14.  Table 3 shows the results of the proposed FLL-CLT, CLBP and CLTP using the Georgia Tech Face dataset. The proposed FLL-CLTC outperformed the CLBP and CLTP texture descriptors and achieved the highest recognition accuracy, reaching 93.21% using FLL − CLTC_S/M /C 3,24 while the CLTC_S/M /C 3,24 achieved 92.33%, the CLTP_S/M /C 3,24 achieved 91.97% and CLTP_S/M /C 3,24 achieved 91.4%. Table 3 shows that the sign part of the proposed FLL-CLTC could not achieve good results and the CLTP S and CLBP_S results are better than the proposed FLL − CLTC_S. The results are due to the nature of the Georgia Tech Face dataset herein the images are in colour format. Converting the images to the grey format may affect the edges of the image contents, which could affect the performance of the FLL, which is an edge-aware smoothing filter. The FLL works well with the magnitude operator of the FLL-CLTC and with the combination of all operators where more details are included in the descriptor.

E. CALTECH PEDESTRIAN FACES DATASET 1999
The Caltech Pedestrian Faces Dataset 1999 was collected at the California Institute of Technology [31].   The proposed FLL-CLTC achieved the best results in most cases using Caltech Pedestrian Faces Dataset 1999 as shown in Table 4. A few cases had CLBP or CLTP achieving better, VOLUME 8, 2020  which is due to the nature of the dataset. The dataset has a noisy background unlike other datasets and their images are in colour format. The intra-variation issue can be seen easily in this dataset.  Figure 16. Table 5 shows the results of the proposed FLL-CLTC, CLBP and CLTP. The results show that the proposed FLL-CLTC outperformed the CLBP and CLTP texture descriptors. The highest recognition accuracy was   Although the proposed FLL-CLTC had the best recognition accuracy results in most cases as shown in Table 5, the CLTP and CLBP could achieve better results in some cases. The ORL face dataset images are in 92 × 112, which is considered smaller in size compared with other datasets. Moreover, the lighting, facial details and facial expressions of the images in the same class also differ. These factors affect recognition accuracy using the sign and magnitude operators especially in the small pattern radius (1,8) and (2,16). The number of training images with pattern sizes may also play a role in recognition accuracy using the ORL dataset. VOLUME 8, 2020   Table 6 shows the best classification accuracy results on all used benchmark datasets using the CLBP, CLTP, CLTC and the proposed FLL-CLTC. Furthermore, Table 6 shows a comparison of some of the previous works for facial image recognition task. The proposed FLL-CLTC showed highest results in all datasets except YALE dataset where Zaaraoui et al. [33] achieved 90.66% while the FLL-CLTC achieved 86.93%.

VI. CONCLUSION AND FUTURE WORK
In this paper, a new texture descriptor, namely Fast-Local Laplacian Completed Local Ternary Count (FLL-CLTC), is proposed. The CLBC is enhanced by adding a threshold during the extraction process to overcome the noisy sensitivity drawback of the CLBC. Then, the FLL is used in the pre-processing stage to increase the discriminative property of the CLTC texture descriptor. The performance for the face recognition task of the proposed FLL-CLTC is evaluated using different facial image datasets. The JAFFE, YALE, Georgia Tech Face, Caltech Pedestrian Faces 1999, and ORL datasets were used with a different number of training images and with different pattern sizes for the texture descriptors. The FLL-CLTC exhibited good recognition accuracy as compared with the CLTP and CLBP. The FLL-CLTC outperformed the previous texture descriptors in all selected face image datasets and achieved the highest results. For future work, we plan to use optimization algorithms to determine the optimum threshold value for each dataset instead of selecting the threshold empirically. Avoiding manual selection will also improve the texture descriptor extraction process and recognition accuracy. Moreover, we will use deep learning instead of the traditional machine learning to achieve high face recognition accuracy to be able to use for real-time video surveillance detection.

ACKNOWLEDGMENT
The authors are grateful to Najran University, Scientific Research Deanship for the financial support of this research. The authors wish to express their gratitude to the anonymous reviewers for their valuable comments and suggestions to improve the quality of the article.
REYAZUR RASHID IRSHAD received the B.Sc. degree from Aligarh Muslim University, Aligarh, India, in 2000, and the master's degree in computer application from Indira Gandhi University, New Delhi, India, in 2010. He is currently pursuing the Ph.D. degree with JJT University, Rajasthan. He is also a Lecturer with the Department of Computer Science, Najran University, Saudi Arabia. He has published many articles in reputed journals and has attended some conferences. His research interest includes web-based applications. He has published many research articles in many ISI and Scopus indexed journal. His research interests include image classification, object recognition, deep learning, natural language processing, and computer vision. He is a Senior Member of IAENG and ASR. He is a member of MBOT. He is also a committee member with many of the international conferences.
BADIEA ABDULKAREM MOHAMMED (Associate Member, IEEE) received the B.Sc. degree in computer science from the University of Babylon, Iraq, in 2002, the M.Tech. degree in computer science from the University of Hyderabad, India, in 2007, and the Ph.D. degree from the Universiti Sains Malaysia, Malaysia, in 2018. He is currently an Assistant Professor with the College of Computer Science and Engineering, University of Hail, Saudi Arabia. He is also an Assistant Professor with Hodeidah University, Yemen. His research interests include wireless networks, mobile networks, vehicle networks, WSN, and image processing. In his research area, he has published many articles in reputed journals and conferences.