Particle Swarm Optimized Fuzzy CNN With Quantitative Feature Fusion for Ultrasound Image Quality Identification

Inherently ultrasound images are susceptible to noise which leads to several image quality issues. Hence, rating of an image’s quality is crucial since diagnosing diseases requires accurate and high-quality ultrasound images. This research presents an intelligent architecture to rate the quality of ultrasound images. The formulated image quality recognition approach fuses feature from a Fuzzy convolutional neural network (fuzzy CNN) and a handcrafted feature extraction method. We implement the fuzzy layer in between the last max pooling and the fully connected layer of the multiple state-of-the-art CNN models to handle the uncertainty of information. Moreover, the fuzzy CNN uses Particle swarm optimization (PSO) as an optimizer. In addition, a novel Quantitative feature extraction machine (QFEM) extracts hand-crafted features from ultrasound images. Next, the proposed method uses different classifiers to predict the image quality. The classifiers categories ultrasound images into four types (normal, noisy, blurry, and distorted) instead of binary classification into good or poor-quality images. The results of the proposed method exhibit a significant performance in accuracy (99.62%), precision (99.62%), recall (99.61%), and f1-score (99.61%). This method will assist a physician in automatically rating informative ultrasound images with steadfast operation in real-time medical diagnosis.

reflection waves which are formed with random energy and 26 this causes generating speckle noise in images.It is inevitable 27 to reduce preserved edges and speckle noise for diagnosis and 28 interpretation of ultrasound images [2].Most of the ultra-29 sound image analysis and filtering methods concentrate on 30 the effect of speckle noise and try to reduce its effects [3] 31 but sometimes the noise makes images distorted, and blurred 32 of LBP in the analysis of texture features.The main advantage of this method was that the method could easily be used to generate the dataset of the synthetic ultrasound image.This method also developed an objective quality assessment for synthetic ultrasound images and this was the core novelty of this method.The whole experiment of this method was evaluated based on the feature of a single technique namely LBP and it was a fundamental limitation of this method.
All of the methods discussed earlier are the classical methods that perform quality analysis of ultrasound images by using various quantitative parameters like MSE, PSNR, etc.In recent years several intelligent works have been introduced for the exploration of ultrasound image quality based on artificial intelligence.The subsequent part of this paragraph presents some of these kinds of methods.Zhang et al. [4] presented a CNN-based image Quality assessment (IQA) model for ultrasound images.To establish the IQA model they had used a deep CNN and a residual network followed by a transfer learning approach.For evaluation of the IQA model two error metrics, LCC and SROCC had been used where PSNR and SSIM were used to evaluate the ultrasound images quality.Based on the result of these measurement metrics they had found that the CNN-based IQA model provided effective results.This model provided an automatic no-reference IQA based on Deep learning (DL) which was the prime advantage and novelty of this method.The proposed IQA technique had some subjective issues during image labeling and this was a fundamental limitation of this method.In the paper [6] a DL-based scheme FUIQA was introduced to assess the fetal ultrasound image quality with the realization of two DL models L-CNN and C-CNN.They had involved 8072 fetal abdominal images from approximately 492 ultrasound videos from which the model L-CNN localized the fetal abdominal region of interest (ROI) and C-CNN evaluated the ultrasound image quality based on that ROI.Later the results of the FUIQA scheme were assessed by three metrics ROI, SB, UV, and suggested that the local phase features were helpful to improve the performance of model L-CNN.An automatic quality control scheme for fetal ultrasound images was the main advantage and novelty of this method.However, the method was applicable only fetal ultrasound images which was a prime limitation of this method.Mostafiz et al. [8] proposed an automatic deep neural network system to detect and reduce speckle noise from ultrasound images.The coalescence of CNN and wavelet features had been used to detect and classify ultrasound images.They attained 98.54% accuracy, 98.19% sensitivity, and a specificity of 98.25% for image classification.They concluded that LDA in noise analysis shows better performance in terms of MSE, SNR, and EPF.This method could detect and remove speckle noise of an ultrasound image by itself which was the prime advantage and novelty of this method.This method was only applicable to the speckle noise of ultrasound images which was the main limitation of this method.A Machine learning (ML)-based scheme was developed in the paper [11] by using the AdaBoost algorithm, to measure the quality of fetal ultrasound images.The automated detection of stomach bubbles and the umbilical vein was also proposed with AdaBoost which takes less than 6 seconds.In the base of accuracy, specificity, sensitivity, and error results they had shown the detection of the stomach was more accurate than the umbilical vein.An intelligence scheme for fetal ultrasound image quality detection was the main advantage and novelty of this method.The whole research was designed based on only one dataset and only for fetal ultrasound images.These were the limitations of this method.All the existing research discussed till now either build techniques to reduce speckle noise or generates a quality assessment approach.However, all most every technique works with only certain types of ultrasound images such as speckle noise, fetal ultrasound image, etc.So, based on the correlation and analysis of previous works, this research decides to build an automatic quality rating scheme for multiple ultrasound quality issues.
The fundamental aim of this research is to rate whether an ultrasound image is normal, noisy, blurry, or distorted.
To build the scheme this research performs feature fusion from an input ultrasound image by using DL and a customized feature extraction approach.After that classification is performed on these fused features to rate the quality of Two customary fuzzy operations namely fuzzification and defuzzification are utilized to construct the fuzzy layer.In any fuzzy process, fuzzification alters natural inputs to fuzzy states.After performing the fuzzy mechanism to those fuzzy states defuzzification alters the consequence in its natural form [30]. Fuzzification and defuzzification are the facile, operable, and feasible mechanisms that most researchers utilize to develop fuzzy logic design.For instance, the authors of the paper [31] developed a fuzzy scheme for analysis and matching the fingerprint.The authors of the paper [32] developed a convolution-based neuro-fuzzy architecture to do the analysis of sentiment from movie clips.In paper [33] the authors proposed a fuzzy CNN structure to predict traffic flow from precarious traffic accident data.In all of these methods [31], [32], the authors follow the approach of fuzzification and defuzzification in between isolated activities.The PSO is a bio-inspired technique that finds an optimal solution from a solution space [13].Nowadays PSO is widely utilized in various DL and ML approaches to increase the efficiency of models through the best optimization.For instance, the authors of the paper [34] utilized PSO to get the optimal parameters for CNN models.In the paper [35] the authors utilized PSO to optimize the parameters of the Support vector machine (SVM).Where, PSO increased the efficiency of SVM to classify different types of plants.The authors of papers [36], [37], [38]

A. DATASET 230
The dataset [49] of this research holds four types of ultra-231 sound images namely normal, noisy, blurry, and distorted.232 Each type has a total of 650 ultrasound images.Thus, a total 233 of 2600 images exist in the dataset.The images of the dataset 234 are collected from various sources on the internet as well as 235 from real-life diagnostic centers.

B. FEATURE EXTRACTION
For feature extraction, this research uses both handcrafted and CNN features.Quantitative features are extracted by using handcrafted features and for this purpose, this research presents a novel feature extraction approach.For CNN-based feature extraction, the proposed method improves an existing CNN architecture by using an additional fuzzy layer with it.
Section 1 and 2 describe the process of feature extraction in detail.Two popular feature selection techniques namely Minimum redundancy maximum relevance (mRMR) [44] and Recursive feature elimination (RFE) [45] are used to evaluate the redundancy of extracted features.

1) QUANTITATIVE FEATURE EXTRACTION MACHINE
QFEM is a customized feature extraction approach presented in this research.QFEM aims to identify the pattern of quantitative features within the images to recognize the qualitative circumstance of that image.Fig. 2 illustrates the working method of QFEM at a glance.QFEM consists of N steps.In 1 st step of QFEM, an image (Y 3 ) is gained from the input image (X) by using the median filter [12] with a 3 × 3 convolution matrix.After that, 15 features are calculated from two images Y 3 and X.These 15 features are identified as F 3 in Fig. 2. TABLE 2 summarizes these features altogether.By analyzing several existing works (i.e- [40], [41], [42], [43]) on quantitative quality assessment 262 metrics of images this research observes most of the methods 263 commonly use these 15 features of TABLE 2. Hence these 264 15 features are selected in this research.In the 2 nd step of QFEM, an image (Y 5 ) is gained from the 266 input image (X) by using the median filter with a 5 × 5 con-267 volution matrix.After that, 15 features (I 0 -I 14 ) are calculated 268 from two images Y 5 and X.These 15 features are identified as 269 F 5 in Fig. 2. Sequentially median filter with n×n convolution 270 matrix gives F n from Y n and X.After using QFEM with 271 T number of steps there exists a total of T×15 features for 272 an image.Although QFEM may apply with any number of 273 steps, the number of steps should be ascertained accord-274 ing to user analysis.Because of getting optimum outcomes, 275 this research uses 8-steps QFEM.For a noisy image of the 276 dataset Fig. 3 demonstrates the mechanism of QFEM for this 277 research.So, by QFEM a total of 8 × 15 = 120 features gains 278 in this research.Fig. 3 shows the cluster-wise visualization 279 of these features for each class of the dataset.Fig. 4    a velocity v i .For the picked evaluation of objective function, the position is remembered by each p i, and this information is stored by a memory pbest i .Memory pbest i is updated every time whenever p i finds a better position.Another memory gbest holds the best position at swarm level for any particle that has visited ever.PSO updates the value of x and v iteratively until an efficient solution is captured.Algorithm 2 presents the working mechanism of PSO.layers including one max-pooling layer at the end of each Conv block and a fuzzy layer at the end of the last maxpooling layer.The rest of the network from the end of the fuzzy layer is defined as the feature classification part.The fuzzy layer is added in between the last max-pooling layer and the fully-connected layer as an additional layer with the existing VGG-19 structure.Two fundamental operations fuzzification and defuzzification are used to build the fuzzy layer.In the fuzzification stage, the output map of the last max-pooling layer is turned up to fuzzy maps by utilizing three membership functions namely Gaussian(G), Triangular(T), and S-shaped(S).For any value q in between p and r with a standard deviation σ , these functions can be defined as following way: 1.While (An efficient solution is not met) 2.
For each p i 3. Update Update the position Use objective function f to evaluate the fitness value of p i 6.

End for 9. End while End:
Different studies have found that the ReLU activation function with the highest value of six (6) helps the network learn the sparse features [46], [47].Thus, we have selected the value of p and q based on the highest value (r max ).The value of p was selected as half of r max , and q was selected as the sum of p with one-fourth of r max .In fuzzy logic, choosing membership functions is a non-trivial problem.The distribution of data is crucial in the selection process.Our research followed a trial-and-error process to choose the mentioned three membership functions.The cost of calculation and the number of parameters for membership functions have also been taken into account.The Gaussian membership function, for example, requires two parameters: mean and variance.It's simpler to see the effect on inference when there are fewer parameters.
In the defuzzification stage each of the three fuzzy maps is turned to crisp values by using the Mean of max (M m ) defuzzification technique.If x j is the max possible degrees in any fuzzy map and N is the occurrence number of x j then M m can be defined as: This research performs fine-tuning from Conv 5 block to the last output layer of the network given in Fig. 3.During finetuning, the hyperparameter of these layers is initialized as the random particles (P) for the PSO.After setting x i and v i for each p i , the proposed CNN model is executed for every p i  Xception [18], and vanilla CNN baseline [48] because of providing contextual outcomes.

C. FEATURE VECTOR
With QFEM this research extracts a total of 120 handcrafted features.From PSO-based fine-tuned fuzzy CNN this research extracts a total of 1536 features from the last fuzzy layer.Thus, by combining 120 and 1536 features a total of 1656 features exist in the feature vector for each image.data set [20], [21].In the paper [22] it is mentioned that a decision tree that contains N leaves partition the feature space into N no. of regions R n , 1 ≤ n ≤ N.So for each tree, the prediction function f(x) can be defined as where C n is a constant appropriate to n The RF is a robust method to handle noise and every DT of RF provides a unit result that assigns each input dataset to the most feasible label [23].

III. RESULT AND DISCUSSION
The prime concern of this work is to develop an intelligent scheme for rating the quality of an ultrasound image.
To develop the system the dataset of this research is partitioned in a ratio of 8:2, this ratio apprises that 80% of data are reserved for system training and the remaining 20% for system testing.All experiments of this work are examined by using 5-fold cross-validation [28] and usual performance measurement metrics of a classifier such as Precision, Recall, F1-score, Accuracy, as well as Normalize confusion  matrix (NCM) are used to evaluate the efficiency of these 408 experiments.TABLE 4 describes these metrics at a glance 409 and Fig. 6 shows the demonstration of different parameters 410 used to find these metrics.the median filter technique as a core component.Although any image filtering approach can be used to build QFEM the median filter is selected based on its suitable results by evaluating several filtering approaches.TABLE 5 shows the overall performance of QFEM for different filtering approaches.This research fine-tuned several well-known CNN models.TP rate against the FP rate.In this curve, the more the 477 value of the area under the curve is closer to one the more, 478 good the classifier is.Fig. 8  proposed model for these techniques.TABLE 23 presents the fold-wise overall accuracy for the performance of TABLE 22. TABLE 22 presents that mRMR provides the best accuracy of 97.54% between mRMR and RFE and this result is less than the proposed model (99.62%).This indicates the 1656 features need no redundancy reduction.
From the analysis of related research as far as we know this is the first DL-based work to rate Ultrasound image quality.Hence this research puts no comparison with existing approaches to evaluate the performance of the proposed method.

IV. CONCLUSION
This research presents an intelligent model to rate whether an Ultrasound image is normal, noisy, blurry, or distorted.To develop the scheme proposed method performs feature fusion from an ultrasound image by using a customized feature extraction approach and a PSO-based fuzzy VGG19 CNN technique and then the RF classifier recognize the quality type of that image from the fused features.Based on the results we have found the proposed approach as an efficient system for ultrasound quality rating by holding an inaccuracy of 0.38% only.In the future, besides the quality rating, we will try to restore the quality of an ultrasound image to normal if the quality is not detected as normal.However, the proposed method will assist physicians to make any decision during ultrasound imaging-based diagnosis.
the input ultrasound image.The customized feature extraction technique of this research is named by Quantitative feature extraction machine (QFEM) and it extracts several quantitative features from an input ultrasound image.For DL-based feature extraction, this research modified the existing VGG-19 CNN model by adding a fuzzy layer to it.
fuzzy VGG-19 CNN model of this research is optimized by utilizing the Particle swarm optimization (PSO) technique.

219
The next part of this paper is allocated in the follow-220 ing way: Section II describes the prime architecture of this 221 research along with the related dataset.Section III presents 222 the obtained results of this research with the necessary dis-223 cussion.Finally, section IV concludes the overall work of this 224 research.225 II.MATERIALS AND METHODOLOGY 226 This section presents the dataset and main formation of this 227 research.Fig. 1 presents the overall formation of this research 228 and section A to D narrates Fig. 1 in detail. 229

FIGURE 1 .
FIGURE 1.The fundamental architecture of this research. 265

FIGURE 3 .
FIGURE 3. Mechanism of QFEM for a noisy image of the dataset.

Fig. 5
Fig. 5 shows the architecture of the VGG-19 CNN model of this research.Like other CNN it has two parts namely feature extraction and feature classification part.The feature extraction part consists of a series of Convolution (Conv)layers including one max-pooling layer at the end of each Conv block and a fuzzy layer at the end of the last maxpooling layer.The rest of the network from the end of the fuzzy layer is defined as the feature classification part.The fuzzy layer is added in between the last max-pooling layer and the fully-connected layer as an additional layer with the existing VGG-19 structure.Two fundamental operations fuzzification and defuzzification are used to build the fuzzy layer.In the fuzzification stage, the output map of the last max-pooling layer is turned up to fuzzy maps by utilizing three membership functions namely Gaussian(G), Triangular(T), and S-shaped(S).For any value q in between p and r with a standard deviation σ , these functions can be defined as following way:
Random forests (RF) algorithm is used to classify the feature vector.Once RF is trained the system can easily rate the quality of an ultrasound image.RF is a popular tree-based supervised ML algorithm that contains multiple Decision trees (DT) for classification tasks.It uses bagging ensemble [19] techniques which improves classification performance compared to other single classifiers.In the original RF model, the classification and regression trees algorithm are used which is a DT variant method that induces DT by recursive, top-down, greedy, and binary partitioning of the of QFEM.To further extend the performance of the proposed model, this research optimizes the fuzzy CNN models using PSO.
also utilized PSO to increase the 201 efficiency of different DL techniques.Thus, this research 202 decides to examine the efficiency of PSO.Following are the 203 principal contributions involved in this research: The proposed method builds an automatic ultrasound 214 image quality rating scheme with a low misdetection rate 215 because of the fusion of features from two techniques 216 QFEM and PSO optimized fuzzy VGG-19 CNN model.217 204 • This research increases the performance of the existing 205 VGG-19 CNN model by adding a fuzzy layer with 206 it and by optimizing the model using the PSO tech-207 nique.Also using the fuzzy layer and PSO technique 208 proposed method analysis the performance of different 209 well-known CNN models.210 • A fancy feature extraction technique named QFEM is 211 presented in this research which performs excellent 212 using only 120 features.213 • • This research generates its own ultrasound image dataset 218 of 2600 images to amplify the proposed scheme.

TABLE 1
presents the sam-236 ple ultrasound images for each class of mentioned dataset.

TABLE 1 .
Sample images for each class of the dataset.
shows 280 that the features of QFEM provide a clear separability among 281 classes.

TABLE 3 .
[14]meters of PSO.and during this execution, the value of x i , v i , pbest i and gbest 352 are updated for the gained results according to Algorithm 2. 353 This execution performs iteratively and the parameter of gbest 354 particle gained at final results is considered as the optimized 355 parameter.After obtaining the optimum parameter proposed 356 CNN model is used as a feature extractor by excluding the 357 classifier part.TABLE3shows the general parameters of the 358 PSO algorithm for this research.To build the proposed CNN architecture this research uses the VGG19 CNN model.VGG19 is selected by analyzing several CNN architectures namely VGG19[14],

TABLE 4 .
Description of performance measurement metrics.
FIGURE 6. Demonstration of different parameters used to find performance measurement metrics.

TABLE 5 .
The overall performance of QFEM by using different filtering approaches.

TABLE 6 .
The fold-wise overall accuracy of QFEM by using different filtering approaches.

TABLE 7 .
The overall performance of QFEM for different steps.

TABLE 9
presents the overall performance of these models.

TABLE 8 .
The fold-wise overall accuracy of QFEM for different steps.

TABLE 9 .
The overall performance of different fine-tuned CNN models.

TABLE 10 .
The fold-wise overall accuracy of different fine-tuned CNN models.

TABLE 11 .
Comparison among different CNN models and proposed QFEM technique.gets lower performance than the DL-based approach but 442 TABLE 11 shows that QFEM outperforms different CNN 443 models.Hence, this research adds a fuzzy layer with different 444 CNN models to improve their performance. 445

TABLE 12
shows the overall performance of different 446 fine-tuned CNN models including a fuzzy layer.TABLE13447 presents the fold-wise overall accuracy for the performance 448 of

TABLE 12 . 449 TABLE 14
presents the comparison among different fine-450 tuned CNN models with and without using fuzzy layers.451Thiscomparison shows that the performance of CNN mod-452 els improves because of the fuzzy layer.Where the fuzzy 453 VGG-19 holds the max accuracy and TABLE12shows this 454 accuracy is 97.46% which is less than the accuracy of 97.67% 455

TABLE 12 .
The overall performance of different fine-tuned CNN models with the fuzzy layer.

TABLE 13 .
The fold-wise overall accuracy of different fine-tuned CNN models with the fuzzy layer.

TABLE 14 .
Effect of the performance of different CNN models due to fuzzy layer and PSO.

TABLE 15 .
The overall performance of different fine-tuned fuzzy CNN models with PSO.

TABLE 15 shows
the overall performance of different PSO-based optimized fuzzy CNN models.TABLE 16 presents the fold-wise overall accuracy for the performance of TABLE 15.TABLE 15 and 16 present the efficiency of different fuzzy CNN models using PSO.
In terms of accuracyTABLE 14, shows the comparison among different fuzzy CNN models of this scheme with and without PSO.TABLE 14 presents VGG19 fuzzy CNN architecture with PSO holds the most compatible accuracy of 98.38%, which outperforms the performance of all individual techniques observed till now in our result and discussion part.

TABLE 16 .
The fold-wise overall accuracy of different fine-tuned fuzzy CNN models with PSO.

TABLE 17 .
Performance of proposed scheme.

TABLE 18 .
The fold-wise accuracy of the proposed scheme.

FIGURE 7 .
NCM of the proposed scheme.The feature fusion of the QFEM and PSO-based fuzzy 470 VGG19 model provides the actual outcome of this research 471 and TABLE 17 presents this result.TABLE 18 presents the 472 fold-wise accuracy for the result of TABLE 17. Fig.7 presents 473 the NCM of this scheme for the result of TABLE 17.

TABLE 19 .
Comparison of the performance of this research with and without feature fusion.

TABLE 20 .
The overall performance of different classifiers for the proposed scheme.

TABLE 20
shows the ROC curve for this This research analyzes several classifiers and from those 487 RF is selected for giving the most preferable outcome.presents the overall performance of different clas-489 sifiers.TABLE 20 presents that the RF classifier provides the 490 highest accuracy of 99.62%.TABLE 21 presents the fold-491 wise overall accuracy for the performance of TABLE 20.

TABLE 20 and
21 prove the justification for utilizing RF in To evaluate the redundancy of the features, this research 495 examines two feature selection techniques namely mRMR 496 and RFE.TABLE 22 presents the overall performance of the 497 493this scheme.494

TABLE 21 .
The fold-wise overall accuracy of different classifiers for the proposed scheme.

TABLE 22 .
The overall performance of different feature selection techniques.

TABLE 23 .
The fold-wise overall accuracy of different feature selection techniques.