Melanoma Detection Using an Objective System Based on Multiple Connected Neural Networks

Melanoma is a common form of skin cancer that dangerously affects many people around the world. Detection of melanoma with the naked eye by dermatologists may be subject to errors. Therefore, the implementation of image processing devices equipped with artificial intelligence can act as a support for the dermatologist in examination and decision making. However, due to the various characteristics of this type of lesions and the presence of noises and artifacts in the images, it is difficult to distinguish melanomas from benign lesions. In this article, we propose a new type of intelligent system which is based on several neural networks connected on two levels of classification. The first level contains five classifiers (subjective classifiers): the perceptron coupled with color local binary patterns, the perceptron coupled with color histograms of oriented gradients, the generative adversarial network (for segmentation) coupled with ABCD rule, the ResNet, and the AlexNet. They are chosen experimentally and consider the following features of melanomas: texture, shape, color, size, and convolutional pixel connections. At the second level (objective level), one classifier (perceptron-type) decides whether the lesion is a melanoma, based on learning-adjusted weight and the decisions at the first level. The second level is based on back-propagation perceptron that provides the final decision (melanoma or non-melanoma). The subjective and objective levels undergo two separate training phases. This approach allows an easier transition of the system from one database to another. This study shows that the use of the objective classifier brings an accuracy of 97.5% and an $F1$ score of 97.47%. These results are better than those of the individual classifier and those of the previous literature mentioned in References.


I. INTRODUCTION
Skin cancer is the most common of all human cancers. After the recent studies, the global incidence of skin cancers increased in recent decades. It is expected that by 2040, the number of skin cancers will increase by 62% compared to 2018 [1]. According to the World Health Organization (WHO) approximately 2 -3 million non-melanoma skin cancers and 132,000 melanoma skin cancers occur globally each year [2]. Although melanomas account for less than 5% of skin cancer cases, they are responsible for 75% of skin cancer deaths [3]. Therefore, skin cancer is a major public health issue, with high annual medical and social costs.
The associate editor coordinating the review of this manuscript and approving it for publication was Shuihua Wang . Although melanoma is one of the cancers that can be prevented, it has been growing exponentially in recent years. The main cause is repeated long-term exposure to UV radiation (from the sun or indoor tanning). Melanoma can progress from a common or atypical nevus that undergoes changes such as shape, size, irregular boundaries, color, or texture of pigmented areas. In almost all cases, it is the type of cancer that occurs in the pigment cells (melanocytes) of the skin [4]. Because melanoma is an aggressive malignant tumor and has a very high risk of metastasis, an in-depth knowledge of the factors that lead to its development is very important.
Early detection increases the patient's chance of survival, as this allows for more treatment options to be tried. Thus, visiting a specialist for a thorough analysis of a skin lesion can save lives. Yet, human visual examination can be subject to errors (lesions showing similarities), leaving patients misdiagnosed. These errors are subjective, sometimes due to a lack of specialist experience. Without technological support, the accuracy obtained by dermatologists using human visual examination for the correct diagnosis of melanoma is 60% [5]. It is proven that the diagnosis of skin lesions using image examination is more accurate than the diagnosis without images [6]. The non-invasive method, based on images acquired by dermoscopic equipment, is generally used in a preliminary computational analysis for melanoma detection. Therefore, automatic detection of malignancy, especially based on artificial intelligence (e.g. deep neural networks) can become an important supporting tool for specialists in diagnosing melanoma [7]. However, the precise diagnosis of skin cancer is made by histopathological examination on surgically obtained samples (considered ground truth).
Clinical images of skin lesions are very different, concerning the following aspects: contrast, shape, size, border, weak resolution, presence of artefacts, hair, veins, texture, etc. This makes it difficult to distinguish between melanoma and non-melanoma lesions [8]. Therefore, an image preprocessing phase is necessary [9]. The most used preprocessing operations are the following: image decomposition on color channels, noise filtering, removing artifacts, contrast enhancement, illumination correction, and segmentation. These operations can effectively improve the prediction of melanoma detection.
There are three main stages in an automated dermoscopy image analysis: the image preprocessing (noise rejection and segmentation), the feature extraction, and the lesion classification.
Different classifiers were used to detect skin lesions in the images, focusing on various aspects such as texture, shape, color, size, etc. Due to the specific nature of learning, the individual classifiers can be considered subjective elements of artificial intelligence. As novelty, we recommend the use of several individual classifiers and the final decision to be based on the correlated interpretation of classifier outputs. In this way, we introduce a more objective approach in the melanoma detection process. The question of how many subjective actors (individual classifiers) must be considered remains open. From a theoretical point of view, this is an unresolved problem. However, based on experimental observations, and as a compromise between cost and performance, we used classifiers based on the analysis of shape, color, texture, and convolutional connections.
The novelty of the article consists of the implementation of a more objective system with high performance, based on artificial intelligence for the detection of melanomas in images. To this end, the system includes several individual, subjective classifiers that exploit different characteristics of melanoma. The subjective classifiers are the following: two individual classifiers based on texture feature (a perceptron combined with color LBP -local binary patterns -and a perceptron combined with color HOG -histogram of oriented gradients), one individual classifier based on shape and color (GAN -generative adversarial network [10] -combined with classic ABCD rule -asymmetry, border, color, diameter) and two end-to-end individual CNNs (ResNet -residual network -and AlexNet). As a new contribution, we use a final perceptron to combine the subjective partial decisions into a more objective one, the final decision. The system was learned and tested using ISIC 2019 [11] and PH 2 [12] databases. The system performance is higher than that of individual classifiers, as well as previous literature.
The next section presents the state of the art concerning automated melanoma detection. Section III deals with methods and materials used for the implementation of the objective system based on multi-neural networks. The experimental results are presented in section IV. The discussions and analysis of the proposed system are contained in section V. The paper ends with a brief conclusive section.

II. RELATED WORKS
As mentioned earlier, due to poor quality of skin lesions images, these need to be improved by preprocessing [13]: eliminate noise, adjust the contrast, remove the artifacts, hair, and segment the lesion. Removing the hair from the lesion images has been a challenge in many works. Specialized preprocessing software has been created for this action, the best known being Top-hat operator and DullRazor [14].
Since the preprocessing stage affects the accuracy of melanoma detection, segmentation is an important step and must be executed carefully [15]. For example, in the case of ABCD rule, one of the most important and difficult problems is lesion segmentation. That is because it can strongly influence the assessment of asymmetry and border [16].
Through segmentation, the lesion is separated from the rest of the skin and can be better analyzed. This is a difficult task because of the low contrast between the lesion and the rest of the skin. In the past, the most commonly used technique to segment the melanoma was the adaptive thresholding [17]. The more recent research of [18] proposes a method based on a deep learning CNN for the efficient extraction of lesion regions. The skin images are preprocessed to reduce possible artifacts because the input images were obtained from standard cameras. This CNN analyzes both the general structure and the local texture-based structure. It can be seen that there are different segmentation techniques such as thresholdbased (the simplest), edge and region-based technique, and artificial intelligence-based technique (most recent) [9]. Different methods and extensive studies [19] were used to generate masks on skin lesions, from simple ones like threshold-based binary images to complex ones based on CNN.
Literature has often reported segmentation accuracy (without classification). The deep learning method is more accurate, however, more expensive than the threshold-based binary image generation. Among neural networks, GANs gave the best results in segmenting melanoma skin lesions [20], [21]. Thus, the authors in [21] compare the GAN method with other neural networks like U-net, SegNet, and ExB and obtain an accuracy of 97% in melanoma segmentation. In this study, we create a mask on the skin lesion using a GAN-based technique. We use this mask for the ABCD rule investigation. The next step after image preprocessing is the interpretation of segmented lesion. Different methods have been used to detect cancerous lesions and especially melanoma from dermoscopic images. The more traditional method (1985) called ABCD rule [22] has been expanded in 2004 into the ABCDE rule (A -Asymmetry, B -Border irregularity, C -Color variation, D -Diameter, and E -Evolving) [23]. These methods of differentiation between malignant, atypical nevi, and common nevi, used by dermatologists, were an effective aid to identify suspicious lesions with the naked eye. The total dermoscopic score (TDS) computed by the extraction of four features of skin lesion images (A, B, C, and D) is used for the lesion classification.
The authors in [24] used texture, fractal, and geometric features to discriminate malignant lesions from benign lesions. The decision is taken based on a voting scheme combining these features.
Recently, new classification methods of skin lesions, like deep neural networks, have been frequently used. A comparative study of different deep learning architecture to melanoma detection is presented in [25]. By transfer learning, deep CNNs have been used successfully in many applications for recognizing shapes and objects in images. They can be pre-trained on certain image bases (such as ImageNet [26]) and then adapted for a specific application. Of these, the most commonly used in the detection of melanomas in skin images were the following [9], [25]: AlexNet, VGG Net, GoogleNet, ResNet, DenseNet, Xception, etc. For example, ResNet -34 was used in [27] to classify melanoma and benign cells, with an accuracy of 92%. The authors in [28] studied deep learning CNN architectures like ResNet-101 and Inception-v3 for classification tasks of skin lesions as benign or malignant. They reported an accuracy rate of 89% for ResNet-101 and 90% for Inception-v3.
Transfer learning and some improvements (fine-tuning weights, replacing classification layer with softmax layer, and augmenting dataset) were used in [29] for AlexNet to obtain good performance in the classification of skin lesions. For the ISIC 2017 database, an accuracy of 95.91 was reached.
The authors in [19] used CNN-based frameworks for three tasks: lesion segmentation, feature extraction, and lesion classification. For lesion segmentation and classification (such as nevus, melanoma, and seborrheic keratosis) an extension of a fully convolutional residual network (FCRN -88), namely Lesion Indexing Network, has implemented and trained. For feature extraction, the Lesion Feature Network framework has been used. The authors in [30] connected the information about texture (features extracted from gray level cooccurrence matrix) with the information about shape and color (ABCD rule) at the entrance of an artificial neural network and obtained a detection of melanomas with an accuracy of 93.7%. Just as [18] and [28], we use information about texture, color and shape, but on separate channels. Thus, our system contains one channel characterizing the general structure (shape, size, color, and border irregularity) and two channels characterizing the appearance of the texture (LBP and HOG).
In our previous work [31] we used a global fusionbased decision system consisting of the following classifiers: two-layer-feed-forward network, GoogleNet CNN, ResNet-101 CNN, NasNet-Large CNN (all deep CNN with transfer learning), and HOG based SVM. The decision fusion is based on a threshold of 0.7 from the total weights. The individual classifier decisions were considered 0 for nonmelanoma (NMe) and 1 for melanoma (Me). Unlike [31], we propose a system based on two classification stages: the first stage uses different aspects of melanoma: texture, shape and color characteristics, and particular features extracted inside deep convolutional networks (end-to-end). These are seen as subjective classifiers. Their outputs are not decisions but probabilities of belonging to the Me class. The second classification stage is a perceptron that uses the outputs of the classifiers from the first stage. Thus, we consider that combining more subjective classifiers into a new final integrator classifier, the decision becomes more objective.
Two databases have been frequently used in research papers for melanoma detection: PH 2 (from Hospital Pedro Hispano) [12] with 200 dermoscopic images (40 with melanoma, 80 with common nevi, and 80 with atypical nevi) and ISIC 2019 (International Skin Imaging Collaboration) with 25332 JPEG images of skin lesions [11]. Unfortunately, these public datasets have several deficiencies, such as small volume (especially PH 2 ), imbalance (the number of non-melanoma lesions is higher than the number of melanomas), hair coverage, lack of consistency in the use of the image acquisition equipment, or different image quality standards. They are difficult to diagnose and, therefore, some researchers used data purification and augmentation to segment and classify melanomas [32].

III. MATERIALS AND METHODS
We propose a system for melanoma detection based on three stages: the image preprocessing, first classification stage (subjective classification), and second classification stage (objective classification). Preprocessing refers to known algorithms for eliminating noise, hair (DullRazor), improving contrast, and data normalization. These algorithms will not be detailed further, the paper focusing on the classification approach.
The first classification stage contains five subjective classifiers working in parallel to detect melanomas: Color LBP-Perceptron, Color HOG-Perceptron, ABCD rule-based classifier, ResNet 101 -deep CNN, and AlexNet -deep CNN. Their outputs, expressed as probabilities with a certain predetermined level of confidence, are used in a final perceptron type module for making the final classification decision (the second classification stage).

A. CLASSIFIERS BASED ON TEXTURE INFORMATION
Information about the color texture of the lesion is of great interest in melanoma detection. From the relevant features for this classification, we chose the LBP color histogram and HOG color histogram. Two perceptrons with backpropagation as in our previous work [33] were used for proper classification on two separate branches in the proposed objective system.

1) COLOR LBP-PERCEPTRON
The LBP operator for texture classification was introduced by Ojala et al. [34]. The LBP discriminator was first applied on a 3 × 3 neighborhood around a pixel and then enhanced in 2002 [35] by extending the algorithm to any spatial neighborhood. The advantages of the LBP operator are the following: high discriminative power, invariance to lighting conditions, simplicity of calculation (allows real-time operation), and the ability to encode fine details. It is well known that the LBP algorithm encodes the local pixel information in a binary code. All the pixels from the neighborhood are compared with the central pixel. If they have values higher than or equal to this pixel, then they are given a value of 1; if not, they are given a value of 0. The values 0 and 1 are sequentially read clockwise. Increasing power of 2 is associated, multiplied with corresponding 0 or 1, and finally summed. The number represents the binary code of the central pixel. This algorithm is applied for all the image pixels and the histogram of these numbers (LBP histogram) is considered as an efficient descriptor of the image texture. Algorithm 1 synthesizes the LBP code calculation for a monochrome (or gray level) image.

Algorithm 1 LBP Code Calculation
1. Create the neighborhood of pixel P x,y : 2. Compare all pixels positioned in N P x,y : 3. LBP concatenation process, LBP binary code: 4. LBP decimal codes: The LBP operator of a monochrome image replaces the pixel P x,y by its LBP code in the decimal format (4). Thus, the LBP output is an image with the same resolution as the input image. From this image, a histogram is calculated by considering equal-width bins (width of 25). We consider that a color channel is represented by 8 bits. Then, the corresponding histogram vector has 10 elements.
Using the histogram concatenation of R, G, and B channels, a characteristic vector (LBP RGB ) of 30 elements is obtained for the color representation of images (5): where l R , l G , and l B , are the elements of LBP histograms on R, G, and B color components.
For skin lesions classification the LBP operator was applied so that, subsequently, the LBP RGB vector will be combined with a perceptron-like neural network. Thus, the LBP RGB vector is used to train and assist the neural network in the decision-making process, as it has proven to be an important mechanism of discrimination in image processing. The perceptron used in this article is a single hidden layer neural network which classifies the input images into two classes: melanoma (Me) and non-melanoma (NMe). The general architecture of the perceptron combined with LBP RGB (namely LBP-Perceptron) for the training phase is presented in Figure 1, where the following blocks are highlighted: Input -Lesion Image (the image of the skin lesion); Color Image Decomposition (R, G, and B components); LBP RGB (Calculation of LBP histogram vectors on each color channel); HLBP RGB (Concatenation of LBP histograms represented by a vector with 30 components); Histogram vectors (Histogram represented by a vector); Weights (The set of perceptron weights w k, k = 1, . . . , 30, corresponding to each bin of HLBP RGB ); Convolution (The product sum of the histogram vector l i and the weights w i ) used to determine the output; Sigmoid (Sigmoid function to determine which defined class was predicted); WA (Weights Adjustments -a process used only in the training phase of the neural network).
In the testing phase, the LBP-Perceptron, noted with P L into system configuration, provides at the output the probability p 1 that the lesion is a melanoma.

2) COLOR HOG-PERCEPTRON
For the Histogram of Oriented Gradient descriptor (HOG), the absolute value of the gradient is calculated at the pixel level. By combining the horizontal and vertical components, it can extract the absolute value (magnitude) of the gradient and its orientation. The amplitude of the gradient indicates the intensity transitions between pixels. On the other hand, the gradient orientation represents the values of the angle at which the intensity transitions between pixels take place.
Considering a grayscale image, HOG [36] assumes the calculus of the gradient magnitude G mag (6) and orientation G dir (7) for each pixel (i, j) -Algorithm 2.
The general architecture of the perceptron combined with HOG RGB (namely HOG-Perceptron) is similar with LBP RGB and is presented in Figure 2  histograms represented by a vector with 27 components); Histogram vectors (Histograms represented by vectors); Weights (The set of weights w k, k = 1, . . . , 27 corresponding to each bin of HHOG RGB ); Convolution (The product sum of the histogram vector l i and the weights w i ) used to determine the output; Sigmoid (Sigmoid function to determine which defined class was predicted); WA (Weights Adjustmentsprocess used in the training phase of the neural network).

Algorithm 2 HOG Descriptor Calculation
1. Calculating gradients (direction x and y) for each image pixel: magnitude (6) and orientation -using arctangent (7), 2. Calculate the gradient components with Sobel operator. 3. Divide the image into cells of specified dimension 4. For monochrome (or gray level) representation, 9 components of orientations (bins) are calculated for equal space of 20 degrees over 0 to 180 degrees interval. 5. For each cell we allocate the gradient magnitude to fill the values in the matrix; 6. Magnitude weighting allocation in adjacent bins for each cell; 7. Histogram normalization.
By histogram concatenation of R, G, and B color channels, a characteristic vector (HOG RGB ) of 27 elements is obtained (8): where h R , h G , and h B are the HOG elements on R, G, and B components.
In the testing phase, the HOG-Perceptron, noted with P H into system configuration, provides at the output the probability p 2 that the input lesion is a melanoma.

B. CLASSIFIERS BASED ON SHAPE AND COLOR INFORMATION
The classification branch based on the particularities of shape and color contains two main blocks: the GAN network to segment the lesion and the ABCD block to evaluate the four features and to establish the classification decision.
Generally, on the segmented lesion, it can see more differences between melanoma and non-melanoma ( Figure 3). Non-melanoma (Figure 3 -left) is characterized by symmetry, regular edges, diameter less than 10 mm, and uniform color. Melanoma is characterized by asymmetrical appearance, irregular edges, larger diameter, and colors which vary (Figure 3 -right). These differences were first observed many years ago and were formalized as ABCD rule [22].
The formulas for asymmetry (As), border score (B), Color variation score (C), and diameter (D) are presented in Table 1 [13].
Given the fact that ABCD rule represents a dermoscopic evaluation algorithm slightly more technical, the solution used in this article for the detection of melanoma from skin lesion images has few changes compared to [22].
For the As calculation, only the shape was taken into consideration and not color or structure. Asymmetry is evaluated in two half-planes obtained by the two main orthogonal axes. A score of 0 was given for symmetry on two axes, 1 for asymmetry on one axis, and 2 for asymmetry on both axes. To evaluate the border parameter, a compactness index is calculated for each of the eight segments of the image and then a total border irregularity index (B) was determined as stated in [13]. One point was added for each color with a percentage of >5% of the area of the lesion to calculate the color variation (C) and instead of different    structural components. Six different colors count in determining the color score: light-brown, dark-brown, black, red, white, and blue-gray. The maximum color score assigned to the lesion is 6, the minimum score is 1. The lesion diameter (D) is used for the last parameter of TDS and a score is established to mark the length of the major axis.
A total dermoscopic score (9) must be calculated for ABCD rule classification. A score higher than 5.45 indicates a high suspicion of melanoma. To be able to associate a probability (p ABCD ) of detecting melanoma by the ABCD method, we impose the TDS normalization by twice the separation index (5.45) of Me from NMe (10). This means a melanoma probability of 0.5 for TDS = 5.45.
In the ABCD method, precise segmentation of the lesion is very important. For image segmentation, we used a conditional GAN-based method like in [36] with two phases: the learning phase and the segmentation phase. GAN has two main adversarial blocks (convolutional type): the generator (G) and the discriminator (D) - Figure 4.
First, in the learning phase, a set of lesion images from datasets (Me and NMe) are used to create a batch of images with mask on the lesions (RM -real masks). Then, a lesion image (LI) is introduced in G to create a fake mask (FM).  Two image pairs are introduced in D: (LI, RM) and (LI, FM). The process is repeated successively, each time changing the weights in G and D, so that FM is as close as possible to RM. Other notations in Figure 4 are the following: O -weight optimizer (one for G and one for D), M0 -null matrix, M1 -unit matrix, and BCE -Binary Cross Entropy criterion for comparison. The process is described in detail in [36]. Second, in the actual segmentation phase, only the generator is used to create the segmentation of the lesion (mask) in the image ( Figure 5). However, D provides a probability of FM approaching the RM (p GAN ). This probability is used together with the classification probability of the module ABCD to obtain the confidence of the classifier (CC) from the corresponding branch of the global classifier. Thus, the inputs of the CC block ( Figure 5) are the probabilities p ABCD and p GAN . The final probability (p 3 ) is the product (11):

C. DEEP CONVOLUTIONAL NEURAL NETWORKS
Two branches of the objective system are dedicated to neural networks, end-to-end learning (transfer learning approach). Based on reference studies [9], [37], and our previous study [31] we chose two efficient deep CNNs: ResNet-101 and AlexNet. In addition, for these neural networks, the input images were resized to a resolution of 224 × 224 × 3 pixels.  Figure 6, contains 101 deep layers and is similar to the typical deep CNN structure, the difference being the construction of residual blocks that allow the use of skip connections or shortcuts to jump over some layers. The use of residual blocks solves the problem of vanishing / exploding gradients. The ResNet-101 architecture is shown in Figure 6, and it is characterized by the repetition (marked as × n) of similar residual units RU A and RU B . These residual units are presented in Figure 7.

2) ALEXNET
AlexNet is the name of a convolutional neural network (CNN), designed by Krizhevsky et al. [39]. AlexNet uses color images as input in the CNN. The images must be  224 × 224 pixels, so they will be adjusted to this size before being used as input data. As output, AlexNet produces two numbers, the probabilities that the lesion belongs to Me or NMe. The sum of all elements of the vector is 1. The AlexNet architecture is composed of eight layers: convolutional (the first five) and fully connected (the last three). Each convolutional layer contains several filters of the same size. These filters are used to extract features from images. An important feature of the model is that it uses the ReLu (Rectified Linear Units) function, which activates neurons in the convolutional  layers and helps reduce training time. The last layer, which uses the softmax function, will be modified to provide a probability of the objects belonging to the two classes: Me and NMe. A representation of the network layers can be seen in Figure 8.

D. GLOBAL SYSTEM
The proposed system has five subjective classifiers: SC 1 , SC 2 , SC 3 , SC 4 , and SC 5 (Figure 9) corresponding to the subjective classifiers described above. Each contains a neural network with a specific task. Two networks of branches SC 1 and SC 2 perform the lesion classification based on texture features (Perceptron P L -LBP and Perceptron P H -HOG). The neural network of the branch SC 3 (GAN) has the task of lesion segmentation. SC 3 uses information about shape and color (A, B, C, and D). The deep neural networks of the branches SC 4 and SC 5 (ResNet and, respectively, AlexNet) achieve the direct classification, based on the end-to-end learning approach. Thus, through the final perceptron P F , the global multi-network system uses information about the shape, color, texture and features extracted by two different deep convolutional networks.
There are four phases in using the system: a. Training phase 1 for the individual classifiers b. Validation phase for the individual classifiers. Confidence coefficient establishing. c. Training phase for the final perceptron. Weights establishing. d. Testing phase. Lesion classification as Me or NMe. When changing the lesion database only the phases 3 and 4 are necessary.
First, the lesion image is preprocessed to obtain a format agreed by all branches of primary classification and improving image representation. The format is 224 × 224 × 3 pixels and the chain of image preprocessing operations is the following: noise rejection (including hair removal), contrast enhancement, and histogram equalization. Each individual classifier is trained separately to obtain the maximum accuracy. Because of the specific architecture and individual training, they have subjective behavior. Thus, a subjective classifier (SC i ) is characterized by an associated confidence coefficient c i . The last level of the objective system has also a perceptron structure, presented in detail in Figure 9 (P F ). The weights w i of P F are adjusted in the final training phase. The dashed line shows an active connection only in this final phase of training (fine adjustment).
The inputs of the final perceptron (P F ) are the outputs of the individual classifiers x i , i = 1, . . . , 5, located on the branches SC i , i = 1, . . . , 5 of the global system. The SC i output (x i ) is the product of the probability p i of lesion classification as melanoma and the confidence coefficient c i given to the classifier in the validation phase (12), The confidence coefficient is considered as the accuracy calculated in the validation phase (13), The hidden layer of P F is a convolutional law of the inputs x i and the perceptron weights w i (14) As for the first training phase, for the second training phase, two classes were defined: Me and NMe. For each class, a series of representative lesion images (LI ) were considered and for each selected image, the probability p i of SC i is computed, Then the inputs x i together with the desired output D o (15), form the training data (T d ) of the perceptron (16).
We consider that the training data is defined as a set of n vectors with 5 elements (16) associated with a scalar. The vector v k represents the subjective outputs (x i , i = 1, . . . , 5) for a training lesion image LI k and the scalar D o (LI k ) is the desired output.
The training process starts with a set of random weights. The dot product between the vector v k and the vector of weights (w j , j = 1, . . . , 5), is computed to determine the network predicted output P o (LI k ) for each training image which is normalized between 0 and 1 using the Sigmoid function (17).
The back-propagation error (Er) is defined as the difference between the predicted and desired output of each training image defined in the training data T d (18).
After computing the back-propagation error, the set of weights must be adjusted. We propose besides using the backpropagation error to also use the derivative Sigmoid function of the predicted output that can indicate how confident is the network about the existing weights (19).
The proposed weights adjustment formula at the training image LI k is presented in (20) where the training data at the subjective level, v k is multiplied with the back-propagation error and the derivative Sigmoid function.
Finally, to reach an optimum for the set of weights based on a specific training set, all this training process is repeated for 800 images and each time small adjustments WA(LI k ) are applied to the weights.

E. DATABASES USED
Our system based on the automatic detection of dermatoscopic images was trained and validated on two public databases PH 2 [12] and ISIC 2019 [11] (Figure 12  between several medical institutions to reduce melanoma mortality. ISIC 2019 is a collection of multiple databases and it is currently the standard source for dermatoscopic image analysis research. Moreover, is the most comprehensive, technically advanced, and accessible resource for digital dermatoscopy. It contains the largest collections of RGB images of various sizes, perspectives, and lighting conditions, and acquired with different dermatoscope types. Most images have associated clinical metadata, which has been verified by recognized melanoma experts. The database contains 25,331 lesion images, 24-bit RGB color, from 9 classes of skin diseases, such as melanoma, melanocytic nevus, basal cell carcinoma, etc.

F. EVALUATION METRICS
In order to quantitatively evaluate the performances of the proposed system for melanoma detection, we use two common metrics: the accuracy (ACC) (21) and F1 score (22). These performance indicators can be calculated from the confusion matrix ( Figure 11  time of the networks was on average 150 minutes/ network [40].

IV. EXPERIMENTAL RESULTS
For system training, validation, and testing we used 1000 images from both datasets ( Table 2). Because the datasets are unbalanced (they contain fewer images with melanomas), their augmentation was necessary.
The augmentation was carried out by four turns 90 degrees (except the validation phase). Thus, we obtained 3,400 images for the four phases of the study required: 2000 -SC training, 200 -validation (establishing the confidence coefficient), 800 -P F training, and 400 -testing the system. The Me and NMe images were balanced. Since the number of Me images equal the number of NMe images, F1 is expected to be close to ACC.

A. RESULTS OBTAINED FOR THE SUBJECTIVE CLASSIFIERS
As can be seen from Table 2 in the training phase 1, we used 500 images (2000 augmented images) from ISIC 2019 database, 250 with Me and 250 without Me (NMe). The LBP histograms and HOG histograms for Me and NMe cases are presented in Figure 12 and Figure 13. The values are grouped in 10 bins (LBP) or 9 bins (HOG) for each color component (R, G, and B) and are normalized.
Some examples of lesion segmentation (binary mask and color mask) with the GAN algorithm for ABCD rule are presented in Figure 14.
To evaluate the SC 3 decision, the TDS score and the probability p ABCD are calculated as in (10). Based on p GAN and p ABCD , the probability p 3 provided by SC 3 is evaluated as in (11). Some examples are given in Table 3. Fig. 15 shows the degree of accuracy and loss corresponding to training and validation for ResNet. One can thus observe an increase in accuracy and, to the same extent, a decrease in losses as one reaches the sixth epoch.   Similar results were obtained for the AlexNet model ( Figure 16). The obtained accuracy was 91.04%.
After the training phase, the validation of subjective classifiers was performed on each branch using 200 images without augmentation. For this purpose, the statistical indicators TP, TN , FP, FN and ACC were considered. Then, the confidences c i , associated with classifiers SC i are established ( Table 4). Note that the best confidence coefficient is given to SC 4 .

B. RESULTS OBTAINED FOR THE OBJECTIVE CLASSIFIERS (GLOBAL SYSTEM)
For the testing phase, we considered 320 augmented images from ISIC 2019 dataset (160 -Me and 160 -NMe) and 80 images from the PH 2 dataset (40 -Me and 40 -NMe). The accuracy and F1 score for individual classifiers (branches SC 1 , . . . , SC 5 ) and the global system are presented in Table 5.     classifier) are given in Table 6. Note that there were situations when the subjective classifiers gave wrong decisions, but the final (objective) classifier corrected the mistakes taking into account the decisions of the other individual classifiers (those with red are given the wrong decision of classification).

V. DISCUSSIONS
In this article, we present a hierarchical structure of classifiers for melanoma detection on two levels. The first level is composed of five classifiers, named subjective classifiers because everyone looks at melanomas from different points of view: texture, shape, color, convolutional transformations, etc. These individual decisions, called subjective decisions are analyzed by the classifier on the second level, which can be considered as an objective classifier. The system performances in terms of accuracy and F1 score are significantly better than for the subjective classifiers ( Table 5). The use of the probability offered by GAN in lesion segmentation does not affect the subjective classification according to the ABCD rule (based on TDS), in Table 6 (SC 3 ). The Table 7 shows that the method proposed in this article has better results than any previous method displayed in Section II.
Remark: In our experiment, the ACC and F1 values are close because the number of Mes is equal to that of NMes. If the number of Mes is significantly less than that of NMes then F1 is less than ACC (see examples from Table 7).
None of the articles propose the combination of complex information (texture, shape, size, color, and combinatorial laws). Some correlate only combinational laws from various CNNs [29] with a majority vote-based decision, others correlate CNN with ABCD rule [43]. A single paper provides similar values of ACC (97%), but refers only to segmentation [20]. As can be seen (Table 7), the simple classification decision based on a majority voting scheme [29] gives poorer results than the more elaborated method based on the perceptron decision [our].
The results confirm that the system offers better performance on accuracy (more than 4.25% compared with subjective classifiers and more than 2.5% compared with the related works), F1 score (more than 4.27% compared with subjective classifiers and more than 3.8% compared with the related works), and time to adapt to a new source (database) of images of melanoma lesions (because only the final perceptron must be trained).
Another advantage of the proposed system consists of two training phases. The first (training phase 1) used only the ISIC 2019 database for the subjective classifiers. The confidence coefficients obtained remain valid for other databases related to the detection of melanomas. An adaptation to another database only requires going through the training phase 2 (final perceptron classifier). In our experiments, images from two databases were used (ISIC 2019 and PH 2 ). Thus, the learning time for PH 2 has been reduced by eliminating the training phase 1.
Four issues remain to be addressed in the future: a) finding a more efficient methodology to determine the evolution (such as, for example, the use of the ABCDE criterion instead of ABCD); b) Answering the question of why 5 subjective classifiers and not more? c) The use of appropriate manual segmentation (currently, these segmentations are subjective in public databases; for this reason the learning of segmentation by neural networks is subjective); d) Finding another user of subjective information (instead of perceptron) which can lead to a more objective decision.
As a next goal, we want to consider the dynamic evolution of the skin lesion over a short time, in order to make a better classification and prediction. Consequently, we want to extend the taxonomy with other types of skin lesions possibly related to a cancer diagnosis.

VI. CONCLUSION
In this article, an efficient solution based on multiple classifiers has been proposed for melanoma detection from lesion images. As a novelty, in choosing the classifiers the following aspects were taken into account: a) the classifiers to be placed on two hierarchical levels: a subjective level and an objective level, b) the classifiers on the first level to exploit different characteristics of the lesions c) each classifier based on characteristics was considered subjective due to specific learning, and d) the existence of a final classifier, called objective, to learn from subjective classifiers and make a final decision that can compensate for certain subjective errors of the first level classifiers. Another contribution was the introduction of a conditional GAN for segmentation of lesions for the classifier based on the ABCD rule. This led to an increase in the accuracy of the classifier. In order to be connected to the final classifier, the TDS score was transformed into probability. An advantage of the proposed two-level system was the faster transition from one database to another (or from one device to another) by simply learning the second level (final perceptron P F ). The proposed method exceeds the performance of other methods presented in references. Thus, the objective neural network achieves an ACC of 97.5% and an F1 score of 94.47%, compared to the highest values reported in the references, 97% for ACC and 93.67% for F1 score.