No-Reference Image Quality Assessment Based on Edge Pattern Feature in the Spatial Domain

This paper proposes a general-purpose no-reference image quality assessment (NR-IQA) method that investigates the image’s structure information from a new aspect, i.e., the characteristic of image edge profiles that depict the directional property of adjacent edge points in the spatial domain of the image. More specifically, we extracted the image’s edge map based on Laplacian of Gaussian (LoG) filtration and zero-crossing (ZC) detection and refined the edge map to be 1-pixel wide. We then explored the edge map by investigating edge profiles’ statistics in a local window with a $5\times 5$ -pixel size. Considering the consensus that natural images consist of directional structures, we found that the spatial distribution property of adjacent edge points can be represented through several edge profiles called edge patterns, which are selected from natural images with a proposed smooth criterion. With the proposed edge patterns and their statistical histogram for the image and the support vector regression technique, we proposed the NR-IQA model based on the edge patterns in the spatial domain, named EPISD. The proposed method has been extensively validated on the LIVE, CSIQ, TID2013, MDID2017, SIQAD, and SCID databases. The experimental results showed that EPISD has a competitive performance with state-of-the-art methods and works stably across different databases.


I. INTRODUCTION
Quality assessment of an image is not a judgment of ''beauty'' or ''realism,'' but an objective measure [1] aiming to predict perceived image quality that is consistent with the human's subjective perception quality. Although the subjective perceptual quality can be obtained by evaluations of a number of people, the method is inherently time-consuming, expensive, and unstable for a practical system. Measuring the subjective perceptual quality cannot obtain real-time feedback for an automatic system.
In practical image and video processing systems, distortions inevitably occur in the acquisition, digitization, compression, storage, transmission, and display stages. In the last three decades, with the rapid development of digital systems, the image quality assessment (IQA) method plays a crucial role, where useful information can be provided to test, optimize, benchmark, and monitor the process.
On the basis of the degrees of using reference image information, objective image quality methods can be classified The associate editor coordinating the review of this manuscript and approving it for publication was Larbi Boubchir . into full-reference (FR), reduced-reference (RR), and no-reference (NR) models. Among the three classes, FR models [2]- [7] are the most matured ones; however, they are hardly applied for most practical application because of unavailability of the reference image. RR models could predict image quality with only partial information of the reference image [8]- [15]. In contrast, the NR models are considered as ''blind'' IQA, where any reference image information is not required. Hence, in practice, the NR models are more useful for applications.
Usually, in NR-IQA model designs, various statistics of images' structural features play key roles. When an image is distorted, the structural features are changed accordingly, and such structural change can be characterized by some specific structure statistics of the image. By the way of calculating feature statistics, NR models can be further classified into training-free method (e.g., [16], [17]) and the machine learning-based method (e.g., [18], [19]). Training-free methods have an inherent generalization ability; however, their performances are weaker compared with those machine learning-based methods at present [20]. Alternatively, with the help of machine learning techniques, such as support vector regression (SVR) and random forest (RF) techniques, statistics of image features can be easily mapped to the image quality index, which assists the machine learning based NR-IQA models to achieve relatively higher evaluation performance. In recent years, the deep-learning based methods that directly map image or image structure to quality index achieve top performances on some single type distortion evaluations [21]- [23], but show comparatively weaker performances on multiply distortion based databases or screen content images databases [21]. In addition, deep learningbased methods generally need a large-scale IQA database to train numerous net parameters, while most existing databases are moderately small. In summary, the NR-IQA models based on statistics of image features and regularized by machine learning are still promising.
There exist various kinds of IQA features for machine learning based NR-IQA methods. For example, BLIINDS-II [24] operates by analyzing the statistics of discrete cosine transform coefficients using the generalized Gaussian distribution (GGD) model. DIIVINE [25] uses a set of neighboring wavelet coefficients modeled by the Gaussian scale mixture (GSM). In contrast, some other approaches directly obtain the statistic sensitive to image quality in the spatial domain. For example, image luminance is processed by BRISQUE [26] with local mean subtraction and local variance normalization, where the mean subtracted contrast normalized coefficients are modeled by GGD and neighboring coefficients are modeled by asymmetric GGD, respectively. Xue et al. [19] proposed joint statistics of the gradient magnitude map and the Laplacian of Gaussian (LoG) response to characterize the image quality. Some models [27]- [29] endeavor to improve local binary pattern (LBP) in the spatial domain. A visual codebook contains several code words, each representing a specific image structure, and the statistics of the codebook can be used for quality estimation. For example, CORNIA [30] classifies normalized image patches whiten by zero components analysis based on a clustering method to create the codebook. The local quantized pattern can also be used for building a codebook [31].
IQA model designs already used structure information represented by edge points. Edge points that mark the image's singularity positions explicitly represent the important spatial information of the image structure, which is relevant to the perceptual quality. IQA models [32]- [34] investigated how the edge points can be used for IQA model design, where the edge points were detected by LoG filtering and zero-crossing (ZC) detection. By exploring how many edge points stay in their original positions in deteriorated image, we proposed a multi-or single-scale FR-IQA model to achieve the state-of-the-art evaluation performance [32], [33]. Meanwhile, EMSQA [35] measures two salient edge attributes which are edge contrast, edge width. GSS [36] only extract gradient direction as an important attribute of the edge. ESIM [37] improves EMSQA by adding edge direction attribute which shows a better performance on four common distortion types (JP2K, JPEG, WN, and GB) and a weaker performance on other types especially in contrast change (CC). GFM [38] employs chrominance information of whole image to add with edge information, which improves those metrics with only edge attributes on the performance of CC, Layer-Segmentation based Compression (LSC), and Color Saturation Change (CSC). In another work, the histogram of counting four-neighborhood points of each edge point was proposed as the feature to build an efficient RR-IQA model [34].
Existing edge-based IQA models explored the quality properties of each single edge point. However, an image structure involves a serial of adjacent edge points that jointly demonstrate the shape of the image structure. To investigate the relationships of adjacent edge points in terms of IQA, we introduced a new way of using edge points. Generally, natural images consist of abundant directional local features in terms of low-level vision [39]. The local feature's semantic information could be revealed by its edge profile that depicts the directional property of the adjacent edge points in the spatial location. When an image deteriorates, the image edge profiles will be changed accordingly. Thus, an IQA model could be built on the basis of investigating the statistics of all the possible image edge profiles. Although image edge profiles would appear in diversity, on the premise that the lowlevel features of natural images are directional in appearance, the edge profiles would be smooth along with their directions VOLUME 9, 2021 and hence exhibit a finite number of distribution patterns. This smooth property is crucial in designing a learned IQA model using such edge profile features. Copious features are unacceptable for building a learned IQA model because existing IQA subjective databases are limited in size, which cannot support a training process with a large scale of features. This study shows that a limited number of edge profiles can be collected from natural images by the proposed smooth criterion described in IID, and the limited number of edge profiles can represent most edge maps of natural images. By collecting all possible distribution patterns of the edge profiles from natural images within a local region (e.g., 5 × 5 pixels), we would have a new way to characterize image local features with the sets of edge profiles as edge patterns in this paper. Fig. 1(a) shows a natural image of the LIVE database, and its enlarged region is shown in Fig. 1(b). Fig. 1(c) is the edge profile of Fig. 1(b), and Fig. 1(d) shows the edge patterns extracted from Fig. 1(c). The normalized histogram of the whole picture is shown in Fig. 1(e), which can be fed into an SVR model to predict the image quality.
In this study, we used the statistical distribution of edge pattern over an image to blindly measure the image's perceptual quality. The proposed NR model consists of two stages: (1) extraction of edge patterns from natural images and (2) quality measurement based on edge patterns. In the first stage, we collected all of the possible edge patterns from the edge maps of several natural images by following the proposed smooth criterion according to the properties of natural images. Meanwhile, we classified the spatial distribution of adjacent edge points in terms of binary value into 65 groups of edge patterns, in which we excluded three groups representing pieces of straight lines with different directions which are not sensitive to quality evaluation for designing the general purpose IQA features. In the second stage, with the collected 62 groups of edge patterns, we represented the image's quality property by the histogram of the occurrence number of every edge pattern. Finally, we employed a learned model based on the histogram feature to predict the image's perceptual quality, which shows a generalization ability across common distortion types.
The main contributions of our paper are as follows. First, considering that the human visual system is sensitive to the image's structure information, we investigated the spatial relationship of adjacent edge points, which is different from the existing IQA models based on structure information [2], [30] and the edge map [32]- [34]. Hence, we produced a new way of using edge points in terms of IQA. Specifically, we first collected several edge patterns that capture most existing edge profiles of natural images in a local region. Second, we proposed the smooth criterion to construct a limited size codebook of edge profiles. The smooth criterion captures the directional smoothing property of the image so that the small number of edge patterns can cover most of the edge profiles of natural images. In this study, the proposed smooth edge pattern feature has been validated to efficiently acquire the structural information of the image and sensitive to image quality. The novel property found from natural images brings a new aspect of introducing a visual codebook that may benefit IQA researchers.
The rest of the paper is organized as follows. Section II introduces the edge patterns and the details of the proposed method. Section III gives experimental results to evaluate the performance of the proposed method. Section IV gives the discussion and conclusion of this study.

II. EDGE PATTERN CONSTRUCTION
In this study, the edge pattern is defined as representing images' local structure based on the ZC map detected from the LoG-filtered image. More specifically, the edge patterns are refined ZC patches sampled from the ZC map. To simplify the representation of ZC patches without losing the edge profile's information, we used morphological operation, that is, the thinning process to abstract the ZC skeletonization. For IQA model implementation, assuming the smoothness of edges in natural images, several edge patterns are extracted from the thinning-processed ZC maps of natural images, followed by smooth criterion.

A. LoG FILTERING AND ZERO-CROSSING DETECTION
The proposed IQA model is based on the edge map. Various edge detection algorithms, such as Sobel, Canny, or ZC detection on LoG-filtered images, can extract edge points. In this study, we chose ZC detection on LoG-filtered image because both LoG filtration and ZC detection are symmetrically sensitive to image edges without any angular favor. This property would help fairly sense the edge structures of both natural images and distortion images. The proposed LoG filtering follows two steps. The first step is the normal LoG filtering, and the filter h is given by where σ is the scale parameter, and x and y denote the spatial locations.
Although LoG filtering removes local correlations of image signals, there still exist contrast correlations in a broader region. The contrast variance in a large spatial scale will affect the selection of the threshold of ZC detection. To address that problem, we further reduced the correlations by applying the Gaussian filter g to normalize LoG-filtered image with the surrounding energy, as shown in Eq. (2), called adaptive normalization (AN) process, which has been validated to be helpful in IQA model formulation [19].
where cc is a small positive constant to avoid numerical instability for small denominator, σ 2 = 2σ ; ⊗ is the linear convolution operator; I is an input of gray image; and L is the output of AN for LoG-filtered image. After the AN process,  selecting the threshold of ZC detection will be consistent regardless of image contents. After applying ZC detection [40] to the image L, we obtained the ZC map for the subsequent process for performing the IQA model.

B. PREPROCESSING OF ZC MAP
The ZC map captures crucial information about the image structure. However, ZC detection would inevitably generate a multiple-pixel-wide edge line in the ZC map, introducing redundancy in representing the edge profile. In this regard, we used a morphological operation, that is, the thinning process [41], to abstract the ZC skeletonization. The thinning process can reduce the edge image's width to 1 pixel, enabling the processed ZC map to demonstrate the simplest connectivity of the edge points without redundancy. The collection of the edge profiles is refined. The thinning process could distort the branch structure, as shown in the example in Fig. 2(a). The branch structure reflects the superposition of two components in light imaging. In this case, the cross-edge point would be erased by the thinning process, which is the black point shown in Fig. 2(b). Indeed, the branch structure should be considered two independent structures, as shown in Fig. 2(c). To this end, we used detection templates shown in Fig. 3 to find the branch structures, and then we refined the edge VOLUME 9, 2021 structure by supplementing the black points and deleting the circle-labeled point [ Fig. 2(b)]. In this study, we called the aforementioned process as branch pruning. After the branch pruning process, we obtained the refined ZC map. In this refined ZC map, the number of edge profile groups collected from natural images dramatically decreases from 23797 to 95, by our experimental statistics, making it practicable to use the edge profiles representing image structures.
The refined ZC map can demonstrate the quality change of distorted images. In Fig. 4, it shows samples of refined ZC maps of a reference image and several distorted counterparts. The images come from the LIVE database. For each image, an identical local region is enlarged and placed beside the image. The DMOS values are given beside each caption. On the basis of viewing these images, it could be found that the refined ZC maps are related to the visual perception of the image quality. Specifically, compared with the reference image's edge map in Fig. 4(a3), other images seem to be more unregular to some extent. In particular, these refined ZC maps distribute differently with different distortion types, which deviate from natural image's statistics. Moreover, the degree of deviation from the natural image's statistics would be employed if natural images' characteristics could be measured, and hence, the image quality could be predicted.

C. CURVATURE RATIO
The refined ZC map outputs 1-pixel skeletonization of edge points. We collected edge profiles from the refined ZC map and then abstracted the edge patterns. The edge profiles were collected from numerous patches sampled from the refined ZC map. The patch has a definite size, for example, 5 × 5. We restricted the sampled patch centered on the edge point. After the branch pruning process, each collected patch only contains one or more straight/curved lines with 1-pixel wide, which are not interconnected. For a collected patch, we only researched the line traversing patch center and neglected the others. However, the missed line will be captured by another sampled patch when one of its points is located at the center of the patch. Hence, all the lines on the map will contribute to the IQA task. In the end, the collected patch contains a single line traversing the center point.
The natural image is abundant in directional structure features. This phenomenon would result in a smooth edge profile in the curvature. To characterize this property, we proposed an index, named curvature ratio (CR), to measure the edge profiles' smoothness. In our edge profile's statistics in each patch, the edge should span most of the patch, which means that the edge enters the patch at a start point (at patch border), crosses the central point, and then exits from an endpoint. In most cases, the exit point is also at the patch border. However, in the case of representing an edge terminal, the endpoint may be inside the patch. To accommodate this situation, we restricted the number of edge points to be bigger or equal to the window size value instead of requiring the endpoint is at the patch border. Fig. 5(a) shows an example for a 5 × 5 patch, where the edge piece contains five-point spans across the patch. We indexed the edge points one by one with variable i from the start point (i = 1) to the endpoint (i = K ), and in this example, K = 5. To calculate the edge profile's direction change, we used the Freeman chain code (FCC) [42]. Fig. 6 shows the FCC's definition that indicates the eight-neighbor position of the next edge point, labeled by S i . Fig. 5(b) shows the FCC value of each point in Fig. 5(a). The CR of the edge profile in a patch is defined as Eq. (3): where K is the number of edge points in the patch, and mod() means modulus operation that outputs the shortest distance between two points on the circle linked by S 1 , S 2 , . . . , S K . Table 1 lists the FCC value of each edge point of the sample shown in Fig. 5. In this case, CR = 2.

D. EDGE PATTERN GENERATION AND SMOOTH CRITERION
On the basis of the aforementioned processes, we can collect various edge profiles from natural images with determined patch size. Normally, the patch size is picked among odd numbers, such as 3 × 3, 5 × 5, 7 × 7, and so on, to ensure a centered edge point. Meanwhile, we calculated the CR of every edge profile. Let us denote the edge profile by P i,c , where i indexes different edge profiles, and c is the CR value. To classify the collected edge profiles, we combined the edge profiles with the same distribution but different directions into one group, denoted by the group number (GN) (Fig. 10). In this study, we called the grouped edge profiles as edge patterns. Moreover, if an edge profile is symmetrical, there will be four elements in the edge patterns: horizontal, vertical, and diagonal at 45 • and 135 • . Otherwise, the group will have   eight members instead. The edge patterns and edge profiles are indeed the same if the grouping process is neglected. To avoid misunderstanding, we alternated the edge profiles and edge patterns. In this paper, we collected the edge profiles from 110 natural images bought with the right of use from a picture company. The 110 images' contents are varied enough to cover most natural scenes (Fig. 7) for some of their thumbnails. The images' size is approximately 500 × 700, which are independent in content to any images of existing subjective IQA databases.
To analyze the edge patterns, we first viewed the map ratio that reflects how much extent a group of edge patterns can represent the ZC map, which is denoted by MR and calculated by Eq. (4): where Z denotes the refined ZC map, j indexes each edge point in Z , N is the number of the edge points in Z , c and i are defined as the same as above, I C denotes the number of the edge profiles for a given CR value, C M denotes the maximum CR value of which the edge profiles are used to represent the refined ZC map, E j (Z ) is an operator to fetch a patch from Z with the center point of j, Like(A, B) is the morphological operator to compare if the two patches of A and B are the same, as shown in Eq. (5).
For natural images, the collected edge profiles have limited CR values. By our experimental results, the CR value did not exceed 2 for the 3 × 3 patch size, 6 for the 5 × 5 patch size, and 10 for the 7 × 7 path size.
On the basis of the 110 natural images, we illustrated the relationship between the map ratio, max CR, patch size, and threshold of ZC detection (Fig. 8). Meanwhile, we listed the number of edge patterns under different patch sizes and CRs in Table 2. From Fig. 8, we can see that regardless of the ZC threshold and patch size, the map ratio can be nearly 100% when the maximum CR is under a certain value. From Table 2, the number of edge pattern group is increasing rapidly as the maximum CR growing. In applying edge patterns for image quality prediction, a large patch size will be helpful because large patch size will capture the diversity of edge patterns' distributions. However, introducing a large number of edge patterns will dramatically increase the burden of calculation. To balance between the representation effectively and computation burden, we adopted the 5 × 5 patch size and set the maximum CR to 4. We called the above maximum CR limitation under the 5 × 5 patch size as a smooth criterion in this study. Under this criterion, we have 65 groups of edge patterns representing the ZC map with a more than 96% map ratio.
To validate whether the proposed edge patterns can effectively represent ZC maps of other natural images except for the 110 natural images, we used the 65 groups of edge patterns in calculating the map ratios for the 103 (29 + 30 + 24 + 20) reference images of the four frequently used IQA databases (LIVE [43], CSIQ [44], TID2013 [45], and MDID2017 [46]). The scales of the LoG filters are 0.5, 1.0, 1.5, and 2.0, respectively. By the result, the average map ratios are 98.34%, 97.19%, 97.83%, and 99.05% over the 103 reference images respectively on the four scales, and under the threshold selections of ZC: 0.98, 0.01, 0.15, and 0.15, respectively. The results suggested that the spatial distributions of edge patterns are consistent among different image contents, and thus, they would become a new effective representation of oriented smooth edge profiles for natural images.
Images' structures exhibit a large amount of redundancy over scales [16], and the spatial distributions of edge points are different at different scales. Hence, the edge profiles of multiple scales distribute differently. As a result, edge profiles' statistics in distorted images are different from those in the natural image over different scales. In this paper, the E j (Z ) of a tested image is extracted on multiple scales to compare with the selected smooth edge patterns. Then, the compared result's statistics can be fed into a learned EPISD (edge patterns in the spatial domain) model to predict the quality score. Fig. 9 illustrates the whole framework of extracting smooth edge patterns from natural images. In this process, the scale of LoG is 1.0, the threshold of ZC is 0.01, and the parameter of AN is 14.44. The calculations of the first three steps in Fig. 9 are introduced by IIA, IIB, and IIC, respectively. For the collected edge profiles, we only kept the ones whose CR ≤ 4. Then, we grouped all selected edge profiles by their distribution into groups. In every group, one member can be transformed into other members by center rotation or flipped. After the whole process above, we extracted 65 groups of edge patterns with CR ranging from 0 to 4 from the 110 natural images. The edge patterns are displayed in Fig. 10. Although we collected the edge patterns under the LoG scale factor of 1.0, the produced edge patterns are generally over other scale factors, by our experimental observation. Fig. 10 shows the 65 groups of edge patterns, in which not all edge patterns are qualified for predicting image quality. Based on our experimental observation which are shown in section III. B, we excluded three groups of edge patterns with GNs 1, 2, and 5(highlighted in Fig. 10). Specifically, the removed edge patterns represent pieces of straight-line edges with different directions, which are insensitive to image quality evaluation. Those edge patterns have large occurrence probabilities, which disturb the training process of building the IQA model. To refine the proposed model, we excluded them from the quality features in yielding the proposed model. Finally, we selected 62 groups of edge patterns to build a quality feature for producing the NR-IQA model. The selected edge patterns have the CR value not exceeding 4. Fig. 10 shows the 65 groups of edge patterns. The GN is placed at the left of each group. The CR values of the edge patterns are listed in the first column.

E. EDGE PATTERN HISTOGRAM AND NR-IQA MODEL LEARNING
We used the histograms of groups of edge patterns as the quality feature of the proposed NR-IQA model. To predict perceptual quality scores based on the histogram features, we used the SVR technique to train the regression model, commonly used in the literature [19], [24], [26], [47], [48]. Here, the prediction model is learned via ε-SVR training between the normalized histograms [H 1 , H 2 , . . . , H k , . . . , H K ] and the corresponding DMOS/MOS [y 1 , y 2 , . . . , y k , . . . , y K ] scores of the training image set with K images. For each image k, the normalized histogram H k is calculated by (6): where f is the count number for edge patterns P i , which are compared with every edge patch E j (Z n ) of the refined ZC map Z n for the test image on each scale n; i denotes the index of the GN of edge patterns, and n denotes the LoG scales, j is the index of each edge point in Z n . Here, we adopted four scales to detect natural image edge points. In this regard, we have 62 × 4 quality features to acquire the perceptual quality of the image. The deep learning method was not introduced on the learning step because the proposed method needs each training picture's subjective value, and all the exiting databases with DMOS/MOS have no enough training samples for deep learning.

III. EXPERIMENTAL SETUP AND PERFORMANCE EVALUATION
The edge pattern histograms are normalized to be the input of SVR for image quality prediction. Specifically, the refined ZC patches E j (Z n ) with the 5 × 5-pixel size are extracted from the tested image of natural scene to match groups of edge patterns P i on scales of 0.5, 1.0, 1.5, and 2.0, where the corresponding thresholds are 0.98, 0.01, 0.15, and 0.15, respectively. Besides, the regression kernel is the radial basis function (RBF) with two parameters C and γ set on each database. Given the edge pattern histograms observed, the trained model EPISD could predict various distorted images' quality score. The MATLAB source code for the EPISD can be downloaded at http://gr.xjtu.edu.cn/web/xqmou/EPISD.
We tested groups of edge patterns' prediction accuracy on subjective image databases to investigate each database's performances and demonstrate their independence and generality across databases. The EPISD is trained and tested separately on the natural scene databases: LIVE [43], CSIQ [44], TID2013 [45], and MDID2017 [46]; and even on the screen content images SCIs databases: SIQAD [49], and SCID [37]. The results showed that for all databases, EPISD are highly correlated with the corresponding subject scores (DMOS/MOS) and stays in the state-of-the-art NR-IQA models and shows a good generalization ability across different image contents.

A. DATABASES AND EVALUATION PROTOCOLS
The LIVE database was published by the University of Texas, consisting of five types of distortion and 779 images generated from 29 reference images. Here, all distorted images were tested with the associated DMOS values in the LIVE database. The CSIQ database was released by the Oklahoma State University, which contains 866 distorted images of six distortion types originated from 30 reference images. Compared with the two databases mentioned above, the TID2013 is the largest database among the four databases. TID2013 database was published by the National Aerospace University and the Tampere University of Technology, which contains 3000 distorted images of 24 distortion types. Those distorted images are generated by 24 natural reference images and one synthetic reference image. In this database, the synthetic image is excluded from the evaluation in the following sections. The MDID2017 database was recently published by the Tsinghua University, which was designed for evaluating IQA on 1600 multiple distorted images generated by a selection of five distortion types and levels. Also, it contains 20 natural images. We used two different sets (600 and 480) of distorted images from the CSIQ and TID2013 databases of four distortion types (JP2K, JPEG, WN, and GB) to follow the IQA work's common experiment setting. These four distortion types are shared in the four databases, and all VOLUME 9, 2021 the distortion types in TID2013 were particularly applied to evaluate the NR-IQA's performance further.
SIQAD has 980 distorted images generated from 20 reference images, and SCID contains 1800 distorted images created from 40 reference images. Although the two databases have 7 and 9 distortion types respectively, we only tested our metric on the four common distortion types which are shared in the aforementioned databases.
We also calculated the following three commonly used indexes for the IQA evaluation: (1) the Pearson correlation coefficient (PCC) after the non-linear matching between the method's value and the subjective value, (2) the Spearman rank-order correlation coefficient (SROCC), and (3) the Kendall rank-order correlation coefficient (KROCC). All the three indexes lie in the range (−1, 1), and the larger absolute value indicates the higher prediction performance.
We applied the LIBSVM package [50] for regression to construct a model between the normalized histograms of trained images and the database's corresponding subjective values. The trained model was then employed to forecast quality values through the normalized histograms of test images. To ensure the model's independence, we made sure that the image contents must be non-overlapping between trained and tested sets. The trained images are distorted versions of 80% of the reference images, and the remaining 20% are allocated to the tested set. Furthermore, the random choice of different contents for trained and tested was repeated 1000 times to ensure the stability between the two sets with different contents; meanwhile, the prediction's median results were recorded. A cross-validation experiment was conducted in each database to determine the values of C and γ for the RBF.
ZC map is determined on the basis of threshold selection. From the Fig.8 (b), map ratios of edge patterns with CR equal to 0 vary a lot under different thresholds. After deleting edge patterns with GNs 1, 2, and 5 for quality prediction, the remaining edge patterns' occurrence probabilities change relative slightly across different thresholds. Acordding to the experiment observation in LIVE database, the SROCC value is not sensitive to the threshold selection. When we randomly selected four thresholds on the four range

B. ABLATION EXPERIMENTS
The ablation experiments of different CR and GNs on natural images and SCIs were listed in the Table 3 and 4. The ablation experiments were performed on images with four common distortion types from the CSIQ, TID2013, SIQAD, and SCID databases and full size of LIVE and MDID2017 databases to follow normal experiment setting for existing IQA evaluations. Table 3 shows that distributions of edge patterns with CR = 0, 5, and 6 are not good features for quality prediction regardless of the image content variations.
According to the ablation experimental results shown in Table 4, 62 groups are more effective than 65 groups. The excluded three groups with GNs 1, 2 and 5 (shown in Fig. 10) are negative to the EPISD model design especially when evaluated on SCI databases. Indeed, the three groups represent pieces of straight lines features with different directions. This kind feature is abundant in images and has very different occurrence quantities across different image contents, which is negative to quality predication.

C. PERFORMANCE ON INDIVIDUAL DATABASE
The following state-of-the-art SVR-based machine learning methods of NR-IQA are introduced as the competitive methods in evaluating the proposed model: Grad&LoG [19] BLIINDS-II [24], BRISQUE [26], GWH-GLBP [47], NFERM [18], and SIQE [51]. Moreover, we also employed some opinion-unaware methods: ILNIQE [52], QAC [53], NIQE [20], and dipIQ [21]. Among them, the dipIQ employs the deep learning method; ILNIQE, QAC, SIQE, and dipIQ have a pre-trained model; NIQE has a training free model; those methods were also repeated for a 1000-time random choice of different contents for 20% tested sets to ensure a fair comparison. The codes of the models are borrowed from either public web sources or authors directly. Moreover, to ensure a fair comparison, we determined the optimal SVR parameters through the grid search for all SVR-based NR-IQA algorithms.
The overall performances of NR-IQA models on six databases were shown in Table 5, and the best two NR-IQA models for each index (SROCC, PCC, or KROCC) were highlighted in bold. In the table, Grad&LoG performs excellently on LIVE, CSIQ, TID2013 and SCID with only single distortion-type images, whereas GWH-GLBP gets the best results on MDID2017 with multiple distortion-type images. Grad&LOG and EPISD show top two performances among all the NR metrics. Despite Grad&LoG shows an obvious better performance than EPISD on SCID database, its performances on SIQAD and MDID2017 databases decline too much. EPISD is the most stable metric with the highest mean value of each three indexes. The satisfactory results of our metric in all the databases show that edge patterns merely recording adjacent edge points' direction information have a great generalization ability for predicting quality of four common distortion types. GWH-GLBP was dedicatedly developed for evaluating image quality with multiple distortion types; SIQE was designed for SCIs; thus, they work not so well in other databases. Table 6 recorded the SROCC values for each distortion type on the five singly distortion databases, where the best two models were also highlighted in bold. Evaluated by the hit count, the proposed method and dipIQ share the second place. EPISD still shows top two according to the mean value and standard deviation. Also, Table 7 presented the NR-IQA results with 24 distortion types in the TID2013 database   generated from 24 natural images to evaluate each model on more distortion types further. From Table 7, all NR-IQA methods indicate poor performances on several uncommon distortions, which are Nos. 14-18 on the TID2013 database. The proposed model is not quite effective when tested on the distortion type of color component, masked noise, and comfort noise. The edge patterns with binary values only reveal the edge profile's spatial information, which are effective on the distortion types which result in spatial structure deviation. However, the proposed EPISD is very effective in MDID2017, as shown in Table 8. Table 8 listed SROCC of all the 1600 distorted images with multiple types combined by five single distortions, which were represented by five different colors on each column, and white color indicated the absence of the corresponding single distortion. Grad&LoG and dipIQ, which show good performances in single distortion types in Table 6, are relatively weak in multiple distortion types that are combinations of five distortion types. On the contrary, GWH-GLBP delivers the best performance on multiple distortion types but weaker on single ones. However, the proposed model keeps in the top 2 spots in Table 8 and the most stable on common distortion types, whether they are single or multiple, as shown in Tables 6 and 8. A one-sided t-test with a 95% confidence level was performed to test the mean values' equivalence, where the SROCC was generated from the NR-IQA models, and the experiments were carried out on the four databases for 1000 iterations. The results were displayed in Fig. 11. In the figure, the ''1'' value indicates that the current row model performs statistically better than the corresponding column model, the ''−1'' value implies that the current column model  performs statistically better than the current row model, and the ''0'' value indicates that there is no statistical performance difference between the current row and column models. On the basis of the listed results, EPISD is statistically superior to four methods on each database. Considering the ''1'' value on the four databases, EPISD and Grad&LOG share the first place with the hit count of 16.

D. DATABASE INDEPENDENCE
The most important properties of the NR-IQA model are independence and generality. In other words, when the model was learned from one database, it could be applicable to test the images in other databases as well. In this paper, 12 different combinations of training and testing pairs were formed by four trained databases and four tested databases.  FIGURE 11. Results of the one-sided t -test conducted using SROCC values of competing NR-IQA models on each database. The ''1'' value indicates that the row model is statistically better than the column model, the ''−1'' value indicates that the column model is statistically better, and the ''0'' value indicates that they are statistically similar in performance.
In Table 9, all SROCC values of NR-IQA models across four databases (LIVE, CSIQ, TID2013, and MDID2017) were presented. These values were also calculated from pictures of any shared distortion types, JPEG, JP2K, blur, and noise, where the best two were highlighted in bold. ILNIQE, dipIQ, and NIQE do not need to take the evaluated database's training step. The mean value results showed that EPISD are top three among the models with the four shared distortion types' training steps. To further validate the method's independence property, we implemented another experiment based on all distortion types, which means that the trained distortion type and test distortion type is not consistent. The results are given in Table 10. EPISD's performance is poorer than ILNIQE but the best among the rest methods evaluated by the hit count number and mean value.

E. COMPUTATIONAL COMPLEXITY
Among all the NR-IQA metrics, the learning-based metric is quite time consuming than pre-trained metric and training free metric. During the feature selection process, our metric's computational complexity is higher than many other learning-based metrics. Given that the experiment was performed on the PC which has a 64-bit, 3.2GHz Intel(R) Core (TM) i5-6500 CPU and 8.0 GB memory, Table 11 listed  TABLE 9. Performance (SROCC) of the NR-IQA models across the four databases in shared distortion types.  computational complexity results among different NR-IQA metrics. In future work, we will improve the step of preprocessing to improve our metric's performance and computational complexity.

IV. DISCUSSION AND CONCLUSION
Among the advanced NR-IQA methods in quality prediction tests, the proposed method demonstrates some remarkable significance. It shows a satisfactory performance when evaluated on the databases with single and multiple distorted images and the most stable performance on the four traditional IQA databases. This highly competitive performance provides convincing evidence that the proposed 62 groups of edge patterns can effectively represent natural image structures to predict perceptual quality. The proposed method identifies edge pixels' joint relationship in a local window with the smooth criterion, which discovers latent characteristics of natural image structures and can be used for building IQA models. In this way, the proposed NR-IQA model performs robustly across different distortion types of images.
Meanwhile, the proposed strategy of using edge patterns to represent the inherent property of natural image structures also provides a new aspect in designing IQA models.
On the other hand, edge patterns merely record orientation information of adjacent edge points with binary values show a good generalization ability across four common distortion types. EPISD with 62 groups of edge patterns also show satisfactory results on SCIs databases. Actually, different distribution of edge patterns can represent different structures, such as geometric information of objects and spatial structural information. All these factors will influence different GNs' performances under different distortion types, which will be investigated in our future work.
The edge patterns explore the image's structure information based on edge point detection. Understandably, distortions with non-structural noise superposition will hamper the detection of edge points. In this case, the proposed method would work unattractively on such distortion types. We investigated the prediction performance of the proposed EPISD on noise-related distortion types in LIVE, CSIQ, and TID2013 databases. However, regarding the prediction performance on noise distortion images in the MDID2017, the proposed method works well on some multiple distortion types containing GN superposition, such as the distortion types 14, 16-20, 22, and 23. This is similar to the case of CC-related distortion images, confirming that the proposed EPISD is specific for acquiring structural changes in the IQA task. In this regard, in future we will focus on how to supplement quality features that efficiently respond to non-structural noise-related distortions and contrast change distortions to the EPISD. In this way, the proposed method would evolve better for various IQA tasks.
In conclusion, we proposed to use specific edge patterns to account for image quality and produced an NR-IQA model. Specifically, followed by LoG filtering and ZC detection, we first collected several edge patterns from 110 natural images with a smooth criterion that can cover most edge points of the image's edge map. Then, we refined the edge patterns into 62 and used histograms of the edge patterns as the image quality feature. With the help of SVR, we proposed the NR-IQA model, named EPISD. The experimental results showed that EPISD stays in the state-of-the-art NR-IQA models in terms of quality prediction accuracy. More specifically, EPISD performs effectively across different databases, which shows the superiority of the proposed edge patterns in predicting image quality. We will further refine the edge patterns and combine other image features to improve the prediction performance for noise and more challenging distortion types in future work.