A Heteromorphous Deep CNN Framework for Medical Image Segmentation Using Local Binary Pattern

Estimating mitotic nuclei in breast cancer samples can aid in determining the tumor’s aggressiveness and grading system. Because of their strong resemblance to non-mitotic nuclei and heteromorphic form, automated evaluation of mitotic nuclei is difficult. This study presents the BreastUNet, a new heteromorphous Deep Convolutional Neural Network (CNN) with feature grafting approach for analysing mitotic nuclei in breast histopathology images. In the first stage, the proposed method identifies probable mitotic patches in histopathological imaging regions, and in the second stage, the proposed model classifies these patches into mitotic and non-mitotic nuclei. For the building of a heteromorphous deep CNN, four distinct deep CNNs are developed and used as the basis CNN model. Deep CNNs with various architectural designs capture the structural, textural, and morphological aspects of mitotic nuclei. The performance of the proposed BreastUNet model is compared to those of state-of-the-art CNNs. The proposed model looks superior on the test set, with an F1 score of 0.95, Sensitivity and Specificity is 0.95 and area under the precision curve of 0.95. The recommended hybrid high F1 score and precision, as well as its excellent generalization and accuracy, imply that it might be used to build a pathologist’s aid tool.


I. INTRODUCTION
Cancer is a disease in which cells reproduce at an abnormal rate and spread to other parts of the body. Cancer starts developing when the human body's starts doing what it is not supposed to do. What happens is old cells do not expire, rather they reproduce substantially and produce new abnormal cells. Therefore breast cancer is a disease which mainly occurs due to irregular cell division of breast tissue which results in a tumor. According to the World Health Organization (WHO) in 2020 alone there were nearly 2.3 million women who were diagnosed with breast cancer. There were also about 685,000 lives lost globally [1], [2]. Breast cancer will have been detected in 7.8 million women in the previous five years by 2020, making it the most frequent malignancy to date. Breast cancer takes more years from women's lives throughout the world than any other malignancy. Women in every country and corner of the globe are affected by breast cancer. It can The associate editor coordinating the review of this manuscript and approving it for publication was Qingli Li . happen at any age after puberty, although it is more prevalent in later life. In the United States, a projected 281,550 new cases of invasive breast cancer will be diagnosed in women by 2021. Second, around 43,600 women in the United States are anticipated to die from breast cancer in 2021 [3]. Moreover Pakistan has the highest rate of breast cancer mortality as well the ones who suffer from it most of their lives as in Asia with 90,000 cases reported yearly and over 40,000 deaths [1]. It is also approximated that in Canada 27,400 women were to be diagnosed with breast cancer and on average 75 women everyday along with 14 deaths [3].
Nowadays, the most competent and efficient method for image classification and recognition is Convolutional Neural Network (CNN) and proves the most reliable and effective CNN models (AlexNet [4], VGGNet [5], ResNet [6] and InceptionNet [7]) for image recognition ImageNet Challenge [5], [8]. These classification and recognition models used natural images for training and testing. Currently, Content Based Image Retrieval models used CNN for feature extraction and for classifiers used these discriminative features for classifications [9], [10]. Different layers play an important role in CNN models to learn features from the images and the subsequent layer such as Fully Connected Layer (FCL) to classify images into a single category or the image may be classified into multiple categories at the same time. FCL and Flatten layers convert the discriminative features into vectors and classify them into different or single categories. In FCL, quantization is used to generate hash code and it gives better performance in image retrieval systems [11].
In early days, models were proposed for segmentation and gave such plausible results. These models used natural images for training, validation and testing. These model have used different hand-crafted features extraction methods such as Fourier transform [12], wavelet-based systems [13], in-variants moments [14], Gabor filters [15] and co-occurrence matrices [16]. The performance of these methodologies are not satisfactory and these all are completely based on a single feature. To overcome the drawback of these methods, Bag of Visual Words (BoVW) concept are proposed to capture the salient features using different methods such as Speeded Up Robust Features (SURF) [17], Histogram of Oriented Gradient (HOG) [18], Local Binary Patterns (LBP) [19], Scale-Invariant Feature Transform (SIFT) [20] and GIST [21]. Due to the vast amount of annotated data necessary in CNN, hand-crafted feature extraction approaches are still utilized and produce good results. While a few authors consider various approaches to solving this problem, such as Radon transform [22] local, most authors gravitate toward CNN models. In fact, the CNN methodologies produce impressive results in every field of vision. One of the most popular dataset of X-ray images (IRMA [23]) is examined for classification and segmentation system [24]- [26]. The unbalanced distribution in the dataset is the most critical point.
The complete flow of the BreastUNet model is depicted in Fig 11, we have modified and developed the model from scratch and a robust methodology is proposed using hybrid CNN and low level sparse features representation. In this methodology, low level features extracted via hybrid CNN and sparse representation increase the performance of on classification segmentation multi-modal medical images.
The second most widely diagnosed cancer in women is Breast Cancer [27]. Breast contains different types of tissues which lie between very fatty tissue and very dense tissue. Each tissue consists of a lobes network. A lobe has a very small tube-like structure which is known as lobules. Milk is produced by the lobules and they are also known as glandular tissue. When growth of breast cells out of control then they form a tumor which is known as cancer. Breast cancer is divided into two categories one is malignant in which cells grow in other tissues or spread to different parts of the body and other is benign in which cells grow but do not spread to other tissues. Any area of the breast might be affected by breast cancer. The two most frequent types of breast cancer are ductal carcinoma and lobular carcinoma. The most common kind of breast cancer is ductal carcinoma. Ductal carcinoma is a kind of cancer that starts in the milk ducts and spreads to the nipple. There are two types of For texture analysis, we use different methodologies of image processing which have been widely used in different applications such as remote sensing, medical image segmentation and classification. The most widely and common methods used in practice are: Local Binary Pattern (LBP) depicted in Figure 1 was proposed by [28], Gray Level Co-occurrence Metrics (GLCM) [29] and Gabor Filters (GF). One of the simplest methods for texture distinguishing and extraction is LBP to employ statistical level and local structure. It has more power to describe features, demands less computations and is easily implemented. Although most of the applications use the traditional LBP for feature extractions along with limitations. To overcome the shortcomings of LBP, different version of LBP has been proposed such as Dominant Local Binary Pattern (DLBP) [30], Gabor based LBP (GLBP) [31], Local Ternary Patterns (LTP) [32] and Completed Robust Local Binary Pattern [33]. The basic mathematical calculation is depicted in equation 1. VOLUME 10, 2022 where N p depicts the neighboring pixels and C is the location of the center pixel. S is a function to find the difference and then compare the value where it is greater than the neighbor pixel or not and assign {0, 1} accordingly. The operations are performed on all neighbor pixels and create a list of 8-bits binary numbers and this number converts it into a decimal number to update the value of the center pixel. Different versions have been proposed to overcome the limitations of the Local Binary Pattern (LBP) algorithm. Garcia et. al [34] explains the different variants of Local Binary Pattern for texture descriptions in details. In this study, we have proposed a novel approach for manual feature extraction. We embed or graft these features with auto features extracted via Convolutional Neural Network (CNN). For auto feature extraction, we explored a novel approach for pertinent features extraction with help of pyramid pooling and Atrous convolution. Before implementation of the actual training model, we improve and refine the weakly label data via pretrained Mask R-CNN and FastR-CNN model. For refinement of weakly labeled data, we propose a Label Optimizer that optimize and improve the weakly segments labels. We achieved state of the art performance using the new method Label Optimizer. Further, these results are forward to the Customized Deep learning model which comprises auto features and manual features.
BreastUNet, a deep Convolutional Neural Network (CNN) based heteromorphous hybrid segmentation algorithm, is proposed in this paper for the study of mitotic nuclei in breast histopathology images. Deep instance segmentation based CNN is first utilized to decrease class imbalance and identify potential mitotic patches, which are then categorized into mitotic and non-mitotic nuclei using the BreastUNet model. In this regard, we created four alternative models to accurately capture the distinguishing properties of mitoses and non-mitoses. The feature and decision spaces learned by the model are employed by a classifier, which acts as a combiner to produce a robust classifier for accurate mitotic nuclei identification. On the standard unseen test set, the proposed BreastUNet is assessed. Our contributions are made in the following areas: • A heteromorphous hybrid segmentation model is presented for segmenting mitotic nuclei in histopathology images of breast cancer. The suggested ensemble utilises the strengths of four individual independent CNN models to increase the segmentation system's resilience and generalization.
• Four bespoke CNN based on the following theories are constructed to capture the morphological, structural, and textural changes of mitoses. Asymmetric splittransform-merge, we Label Optimizer LO) are proposed for weakly annotated mitotic cells and also mitigate the False Positive and Negative issue. For feature grafting, we propose GLocal Pyramid Pattern (GLPP) for pertinent feature extraction and to merge into the middle of proposed CNN model using Custom Layer and use customized deep residual network block and residual learning are some of the techniques.
• We compare state-of-the-art CNNs models such as AlexNet, SqueezeNet, Inception, DenseNet, UNet, VNet, UNet ++ with the proposed heteromorphous BreastUNet which significantly improves classification system precision. The following is how the paper is organised: Section 2 offers a summary of the literature work. Section 3 summarizes the materials utilized for the proposed methodology procedure, while Section 4 discusses methodology and implementation specifics. Section 5 brings the paper to a close.

II. RELATED WORK
Jonnalagedda et al. [35] have worked on the magnification invariant classification of breast cancer. They have tried to highlight and resolve the biggest incompetence of histopathological image classification. Most of the research work in this field has different models for classification for different magnification powers of image. In this research paper, they tried to overcome this issue by presenting a single model for all the magnification levels. They used a Multi View Convolutional Neural Network named as MVPNet which analyzes images with both high and low magnification powers. Different paths are available for different magnification levels respective of their level. They also introduced a data augmentation technique named Nu-View. In this augmentation they focus on clusters of nuclei in low magnification images and convert it into an image with size equal to the rest of images and add it to the training set. This helps in the way that nuclei cluster is the most common bio-marker of malignancy detection and converting those patches into full size images helps as it increases the amount of attention it will get while training the classifier.
According to Nurmaini et al. [36] breast cancer is the major cause of death among females all over the world. It build a classification model Multilayer Perceptron (MLP) and 10-fold cross validation to classified the cancer is the recurrent and no-recurrent using the dataset given by University of Medical Center, Institute Of Oncology, Ljublijana, Yugoslavia. It contains 286 data with 2 classes (201 No-Recurrences, 85 Recurrences) and 10 attributes. Simple neurons (perceptrons) are included in MLP. The MLP architecture is a feed-forward neural network layer therefore neurons are sorted in consecutive layers and the information flows from input to output layer using hidden layers in a single direction. The technique used for classification of breast cancer in MLP calculates 69.9301% evaluation rate with 96.5% accuracy level at the 8th evaluation from 10-fold cross validation. This technique calculates a 69.2308% evaluation rate at 9th evaluation from 10-fold cross validation. And at 10th evaluation this technique calculates 64.6853%.
According to Kahya et al. [37] a vigorous amount of effort has been put in automated detection of breast cancer through histopathological images to classify into Malignant and Benign. In the extraction of features there may be outliers in the values and the outliers affect the accuracy of the system. The best way of eliminating the outliers is the smoothing of the features matrix. By integrating penalised logistic regression with the smoothed features matrix, the goal of smoothing features and providing high classification accuracy of histopathology images is achieved. For cancer classification, the adaptive penalised linear regression approach is applied. Using this technique, they got the maximum accuracy of 93.496% at 200× size of image. The findings show that APLR is a promising approach for medical image classification, tumour medical diagnosis, and other sorts of high-dimensional classification data in the biological area.
According to Xie et al. [38] this handout related to automated counting of cells and the images that are classified under the microscope. Detection based counting and Density Estimation are two ways for cell counting. They use other ways to show that cell detection is in favor of cell counting problems. As this is a supervised learning problem, they fling the cell counting problems that try to know about the mapping between a density map and an image. They use the Convolutional Neural Networks (CNNs) to solve the mapping problems. Which has become a regular tool of the computer vision community. If they analyze biomedical images the CNNs are very popular for it. CNNs give excellent results to real images as compared to Networks if both models are trained only on synthetic data.
To overcome the errors of density map they modeled Fully Convolutional Regression Networks (FCRNs). They are facing problems in detecting and dividing the individual cells so they also modeled the cell counter from density estimation and used the CNN for the detection of the cells. The FCRN uses a density map for the image that is of self-assertive size. Synthetic data is used to train the model they build i.e. FRCN and the results show that synthetic data is much better than the images after the biological experiments but according to their vision the results can be more efficient if the images of biological experiments are finely tuned. They analysed the performance and found that 95.7 percent of synthetic data and 93.2 percent of microscope images from real biological investigations performed well.
According to Cirecsan et al. [39] the major concern to this paper is detection in microscopy images based on counting. Normally, the count is performed manually by histologists. Mitosis is a stained image structured of Hematoxylin and Eosin and can be of dark blue spots [40]. A DNN method is used as a powerful pixel classifier. It is implemented on raw RGB data sampled from mitosis. In this process no, human input is needed. Image classification is now digitized in the biomedical field. After classification class is assigned to the image either it is mitosis or non-mitosis. Max-pooling (MP) layers, rather than sub-sampling layers, are one of the architectural distinctions between our DNN and earlier CNN. As a consequence, this project was assessed, and the outcomes were correct. It surpasses all competing algorithms with a modest computing effort: processing a 4MPixel image on a normal machine takes only a few minutes.
Further, there have been few methods that used an end to end pixel wise classification scheme for the detection of mitosis. Fully Convolutional Net developed by Ciresan et al. based on a max-pooling base which performed pixel wise classification. The proposed model was trained on the MITOS12 dataset and it was tested on an unseen dataset and achieved an F1-Score of 0.78. Nevertheless this approach is quite slow and computationally exhaustive [39].
Moreover the MITOS14 dataset only provided centroid annotation rather than morphological information of the mitosis which thereby increased the difficulty level of mitosis detection. For the dataset's complexity, Chen et al. developed a cascaded CNN technique. They utilised to track down mitoses in the Fully Convolutional Network. A fine discrimination model was employed in the second phase to eliminate false positives and improve accuracy. [41].
Wahab et al. investigated a two-phase classification technique for mitosis detection to address the class imbalance problem (a classification problem in which the distribution of instances across known classes is skewed). To identify the tough cases, a CNN was trained on the original dataset in the first step. However, in the second phase, the dataset was improved by undersampling negative cases using histogrambased k-means clustering on blue ratio images, as well as supplementing the dataset with hard negative examples for mitosis detection [42].
Wahab et al. [43] proposed a mitotic classifier based on transfer learning. For mitosis identification, they first used a bespoke pre-trained FCN (Fully Convolutional Network). Further, the second phase explained the first phase's results and predictions by allocating the output to another CNN (Convolutional Neural Network) that was a combination of AlexNet and bespoke layers. The winning mitotic detection technology used a two-stage detection strategy to enhance the F1 score in the TUPAC16 challenge [43].
Another issue arose, and that was the use of a dataset that was only moderately updated for deep learning model training. Li  The F1 scores of 0.562, 0.673, and 0.669, respectively, were achieved using these methods on the MITOS14, AMIDA13, and TUPAC16 datasets [44], [45]. The described method employs circular labels to display the mitotic zone, which eliminates the possibility of overlapping with the non-mitotic region [46].
In computer vision, region-based CNNs (R-CNNs) have demonstrated high performance for object detection tasks. Li et al. used VGG16 backboned quicker R-CNN to segregate the areas with a high chance of being mitotic, which were then clarified by assigning the predictions to another deep neural network to eliminate false positives [47].
The authors dealt with the complexity of mitosis by first using R-CNN to identify locations that are likely to be mitotic. By allocating the initial phase findings to the ensemble of ResNet and DenseNet, they were able to segregate non-mitotic areas from the chosen regions. On the MITOS12 and MITOS14 datasets, the suggested technique received F1-scores of 0.858 and 0.691, respectively [6].  For handling the above mentioned issues, we use cross validation technique, it randomly re-sample the dataset to assess predictive models. In cross validation, we assign value to K, it randomly distributes the entire dataset into K-partitions, e.g. k=5, it becomes 5-fold cross validation. K-Fold Cross Validation is used to analyze the forecasting of deep learning models especially on unseen data. The selection of K can cause variance and bias of the predictive model. The depicted Figure 3 and Algorithm 1, explain the crux of cross validation, where K is folding value, and D shows the dataset, f i present single fold (containing portion of dataset), τ shows testing dataset, T contains training dataset, M depicts training model, A depicts accuracy of each training folds and F depicts accuracy function. We equally and randomly distribute the dataset into different K-Folds to train the model on each item during the training model.

B. PREPROCESSING OF HISTOPATHOLOGICAL IMAGES 1) STAIN NORMALIZATION
It is used for mitigation of various color variations proposed by Macenko et al. [48]. Equation 2 has expressed the Stain Normalization. The main reason to use Stain Normalization is histopathological images are obtained from different Laboratories with various acquisition protocols resulting in noise and non-uniformity in color saturation. Before assigning to a deep learning model, we should normalize the complete datasets. The results of Stain Normalization are depicted in Figure 4.
where ε -ratio of intensity express the measurement of absorbance of histopathological images, ψ shows their saturation values, ν show the stain vector. Where µ and σ depicts the mean and variance of RGB values of an image andh shows the stain normalized image.

2) REFINEMENT OF WEAKLY LABELED DATASET
In this study, we have proposed a novel framework for optimization of weakly labeled images of Breast Cancer datasets MITOS12, MITOS14, TUPAC16 and BreakHis [49]. For analysis of Haematoxylin and Eosin (H&E) stained breast cancer histopathology images, a Hybrid CNN with GLocal Pyramid Pattern (GLPP) technique is developed.
To reduce colour fluctuation, numerous augmentation and colour variation approaches, such as Stain Normalization, are used at first. Deep segmentation models such as Mask R-CNN are used to further segment a weakly annotated breast cancer dataset [50] and Fast R-CNN [51]. We use pre-trained MaskR-CNN and FastR-CNN models to learn the morphology of breast cancer tissues.
We have used different datasets of mitosis such as MITOS12, MITOS14 and TUPAC16. These datasets have weak annotations and manual annotation is extremely time consuming and error-elongating for pathologists. The where coord(ϕ) is the slice of an image and ϕ ← R and ϕ m is mitotic or malignant region and ϕ o is the background of a slice. H(ϕ(x)) is a Heaviside function for smoothness. VOLUME 10, 2022 The label-optimizer algorithm depicted in equation 3 annotates the False Positive samples with help of generated mask and coordinates. In our proposed label-optimizer algorithm, it takes two parameters (patch or sample of False Positive image and predicted coordinates of a mask). Using the input coordinates, it extracts or contours the patch from the False Positive images and compares them with the True Positive predicted values. Figure 6 examine the results of proposed Label Optimizer algorithm with MASK-RCNN result. The backbone of the MASK-RCNN is ResNet and Feature Pyramid Network (FPN) that extract the features from 4 different scales and then find the exact region by using Intersection over Union.
Using the above procedure, we optimize the weakly labeled images and get plausible results. The optimized results are passed to the proposed CNN based model for auto-feature extraction and to GLocal Pyramid Pattern (GLBP) for manual feature extractions. These manual features graft with proposed CNN model to remove the blindness of CNN.

C. GLocal PYRAMID PATTERN (GLPP)
We have proposed a GLocal Pyramid Pattern for texture recognition in Breast Cancer datasets. In this method, we split the image (X,Y) into x×y vectors for each smaller patch using of sliding window s×s, where x = X s and y = Y s . We obtain S = [x 0 , x 1 , x 2 , . . . , x w,h ] into a 1-D Vector. We repeat the above operation upto K times or until the patch size is equal to original image size. Suppose the image is (X×Y) and the image smaller patch is x×y. During up-sampling, we extract the Binary features in a clock/anti-clock way and change the initial point in an iterative way for a single patch. In an iterative way, we get a vector of different features and count those features for analysis whose average is maximum. We concatenate all the features of different scales at F GLPP . This operation is performed for all patches in a similar way. The proposed mathematical calculation is depicted in equation 5 and graphical representation is depicted in Figure 7 and the architectural calculation of GLocal Pyramid Pattern is depicted in Figure 8. where N p depicts the neighboring pixels and C is the location of the center pixel. S is a function to find the difference and then compare the value where it is greater than the neighbor pixel or not and assign {0, 1} accordingly. C ⊕ (p i ) are the clock-wise extracted features of different patterns from Local to Global via pyramidal way and C ⊕ (p i ) are anti-clockwise features. F are the summation of maximum clock and anticlockwise features. These features are saved in a Vector for grafting with CNN features to improve the results of CNN. We inspect the proposed GLocal Pyramid Pattern algorithm via Class Activation Map (CAM). The CAM inspects the image and highlights those portions of an image where features are extracted for analysis. The Figure 9 visualizes the extraction feature patches of Breast Cancer datasets.
UNet is one of the most widely used deep learning architecture for segmentation purposes. This network consists of an encoding and decoding scheme of layers. Basic Convolution, max pooling and ReLu layers are used in the encoding  Based on UNet architecture, we have proposed deep residual CNN for Breast Cancer segmentation in this study. The proposed deep residual CNN model based on UNet is called BreastUNet which is depicted in Figure 11. Section III-D explains the architectural details of this model.

D. BreastUNet MODEL
Most widely used efficient neural network architectures have applied Depthwise Separable Convolutions and we use them in our proposed BreastUNet. The main purpose of this layer in our study is to figure-out the local features on low magnification images. It splits simple convolution into two separate VOLUME 10, 2022 layers. It applies a single lightweight convolutional kernel on each input channel and the second layer has 1×1 convolution for pointwise calculation and creates new features. It reduces the computation time compared to traditional layers.
Suppose,we have an image of size (X×Y×C), where C is the number of channels and in depthwise the convolution kernel is k×k. The number of input channels is C and the number of output channels is C out . The output feature map (F) of the traditional convolution is depicted in equation 6 and pictorial representation is depicted in Figure 10 and the total trainable parameters are calculated by k×k ×X ×Y . The Depthwise Convolution is depicted in equation 7 and total trainable parameters are calculated by k×k×X+X×Y.
Another well known concept ResNet (Residual Convolution) is used to derogate the vanishing gradient issue. In ResNet depicted in equation 8, the original input is added into the output of the conventional convolution block. In simple Convolution the container of a learner function is Y, but in ResNet the container is F(X). The proposed ResNet is inspired from the traditional ResNet and we replace the traditional Convolution layer with Depthwise Separable Convolution and the activation function is mish in ResNet Block. where χ is an input image κ is a kernel size of dimension f 1 and f 2 .
Equation 10 contains the output of the above equation 9 where we flip the kernel horizontally and vertically during convolution and bias value as a b.
υ l i,j is the input vector of a layer during convolution and l i,j is the output layer, after convolution and f(·) is an activation function of an activation layer.
For calculation cost (λ) of network the depicted equation 12 express the mathematical formulation to find cost value of a network, where a fc presents actual forecasting cost, n fc is network predicted cost.
Mostly the above equations are used during CNN training, so for integrating manual features within Convolution layer, we introduce a novel concept of custom layer convolution which comprises automated feature extraction and grafting of handcrafted features to improve the efficacy of classification of breast cancer. In the depicted equation 13, F T GLPP presents the ingestion of handcrafted features in automated features.
We apply backpropagation using chain rule with subtracting of our handcrafted feature before finding the ∂ derivative. For finding gradients on individual weights chain rule is applied. The depicted equation 14 show the subtraction of handcrafted features that are place in a single vector (F T GLPP ).

IV. EXPERIMENTS AND DISCUSSION
To present the demonstration of our proposed BreastUNet model, we have tested on three different datasets TUPAC-16, MITOS14 and BreakHis. These consist of various histopathological 2-dimensional images of multi magnification levels. We use basic image processing algorithms and pyTorch on a single graphics processing unit with RAM 8GB and Nvidia GeForce GTX 1050 Ti.

A. EVALUATION METRICS
For evaluation metric and quantity analysis, we use multiple performance metrics such as Accuracy (Acc.), Sensitivity (Sen.), Specificity (Spe.), F1-Measure (F1), Dice Coefficient (DC), Jaccard Index (JI) and Tanimoto Coefficient (TC) and Diagnostic Odds Ratio (DOR). Accuracy, Sensitivity and Specificity equations are depicted in the following equation 15

B. EXPERIMENTAL RESULTS
We investigate our proposed model BreastUNet on three different datasets such as TUPAC-16, MITOS14 and BreakHis.
In the study, we used MASK-RCNN with a proposed Label Optimizer for segmentation of weakly label mitotic cells. Table 2 shows the results of validation and testing. The first column of figure 6 Table 3 and Table 4 depicts the testing and validation results of different State Of The Arts CNNs pretrained models. After getting the results from pretrained models as depicted in Table 3 and 4, we extract manual features from MITOS14, BreCaHAD and TUPAC-16. For texture feature extraction, we proposed a novel approach Local to Global Pyramid Pattern (GLPP) which is completely based on Local Binary Pattern. In this approach, we fragmented the image into x×y vectors for each smaller patch using of sliding window s×s, where x = X s and y = Y s . We obtain S = [x 0 , x 1 , x 2 , . . . , x w,h ] into a 1-D Vector. Repeatedly,we calculate the above operation upto K times or until the patch size is equal to original image size. These operations are performed clockwise and anti-clockwise and get those features who have maximum sum. The maximum sum of smaller vectors are attached with a large vector to create a 1D features vector. These features vector is grafted with after Convolution layers. The feature grafting is not a simple concatenation approach. It creates multiple exceptions to concatenate manuals with CNN features because of the size of 1D vector and Conv2D convolutions. To tackle these issues, we convert the 1D feature vector into 2D tensor and compare the size with Conv2D. During comparison of 2D tensor of manual feature with Conv2D output, if the size of 2D tensor is less, we adopt four different approaches (1) we pad the 2D tensor with zeros (PwZ), (2) we pad the 2D tensor with ones (PwO) (3) we pad the 2D tensor with boundary vector (PwB) and (4) we pad the 2D tensor with maximum sub-vector (PwSV). The results of four different approaches are depicted in Table 5. The manual features padded with Zeros and Ones results are less appraised as compared with padded with Boundary vector and padding with maximum sub vector. The main reason is that with the first two approaches, it adds Zeros and Ones with extracted features and during convolution these Zeros and Ones decrease the Specificity, Sensitivity and F1-Measure. The manual features which are padded with boundary vector and maximum sub vector depict plausible results because it is bound by rich information and the tensor boundary have strong correlation with remaining values of tensor.
The performance diversity and miss-classification shows in different pretrained models depicted in Table 3, 4 and analyze the variations of False + and False − . Figure 12 depicts the False + and False − examples of mitotic cells by the most widely used pretrained models; however these results are considered especially in our proposed models grafting with manual features. For verification of our proposed model results, we use Class Activation Map (CAM) to visualize the results of mitotic cells. It is used to highlight those features who contribute to the final output of deep learning models. The high activation contributes more in final classification and it shows red or closer to red. We evaluate our proposed model cognitive process and detection power in several orientation images and the results are depicted in Figure 13 and show effective naturalistic cognition.
For mitotic classification problem, we have used traditional pretrained CNN models on TUPAC-16, MITOS14 and BreCaHaD and show exemplary results. We pick the top pretrained model (UNET ++ ) customized version of UNET. The manual extracted features are grafted with UNET ++ . Table 5 depicts the comparative results of different variations of GLPP and shows exemplary execution on datasets. The GLPP pad with maximum sub-vector and boundary values depicts plausible results and shows better performance based on F1, Precision and Recall. Precision and Sensitivity is important for imbalanced datasets. It summarizes the positive predicted mitotic cells and False + decrease the F1 values and can impact the prognosis treatment and assessment of the patients. In an unbalanced data set, a small number of false positive results can drastically minify the accuracy of the diagnostic system and reduce the F1 score. These errors can affect the patient's diagnosis, prognosis and therapy. According to the precision-based study, the suggested BreastUNet (precision: 0.95) is good at distinguishing positive cases that genuinely belong to the mitotic class, and it improves by 7% and 9% over the top performing base model and state-ofthe-art CNN, respectively. Similarly, Table 5 indicates that when compared to base classifiers and state-of-the-art CNNs, the proposed approach ''BreastUNet'' dramatically lowers erroneous mitotic cells. Figure 13 depicts an example of challenging non-mitotic samples that are misclassified by basic classifiers; nevertheless, the proposed ''BreastUNet'' accurately classifies these mitotic cells. On the test set, precision and ROC curves of our proposed ''BreastUNet'' model and exiting CNNs are illustrated in Figure 14, 15 and 16. Both curves are used to graphically depict the efficacy of a classifier at various thresholds. However, when the cost of misclassification is different for positive and negative classes, the ROC curve might produce implausible findings for the unbalanced dataset. Even if the model has extremely poor accuracy, it is likely to produce a high AUC. The precision curve, on the other hand, concentrates on the minority class and displays the model's precision as well as the detection rate. The results are compared using the TUPAC16 auxiliary dataset, which is freely available on the challenge website. The findings indicate that the proposed Hybrid BreastUNet model outperforms existing approaches in terms of F1 score. In this paper, we propose a hybrid Breast-UNet model, a novel hand-crafted-based approach for rapidly automating the mitotic detection job. Further, we addressed how the use of several pretrained CNN models might aid in the better investigation of mitoses structure. To achieve knowledge-based undersampling of non-mitosis objects, the   proposed BreastUNet initially uses instance segmentation and detection illustrated in Figure 13. Despite the fact that this stage removes a large number of negative class objects, it contains a large number of false positives that look like VOLUME 10, 2022 mitosis. To exclude them, a sophisticated and exact categorization system is required. As a result, we painstakingly constructed a unique CNN-based model to capture diverse patterns of mitosis during different stages, bearing in mind the heterogeneity of the data. In CNNs, we have combined the concepts of area homogeneity and invariance, asymmetric split-transform-merge, dilated convolution, attention, and residual learning from an architectural standpoint. We have shown that the proposed model is sufficiently diversified depicted in Figure 12 and Table 5. Furthermore, the accuracy of the CAM response in the image's target region implies that the created CNNs are interpretable and capable of performing context-aware learning even without pixel-level labels depicted in Figure 9. Our proposed method differs from the majority of existing methods, which rely on basic Convolutions, Skip Connections, and a mix of pooling layers. We create a Local Optimizer for improving poorly labeled images and a Local to Global Pyramid Pattern (GLPP) for extracting pertinent features, which we subsequently grafted into deep heterogeneous stacking Layers. According to the results of the empirical assessment, aggregating feature and decision spaces improves the detection phase output significantly depicted in Table 5. The suggested approach retains a decent detection rate with a little divergence in accuracy illustrated in Table 3 and 4, whereas the proposed technique's lowest response was on flipped images. Because histopathological images are orientation independent, we intend to investigate the notion of pretrained CNNs in the future to improve model robustness. The improved F1 score (0.95) when compared to existing approaches shown in Table 4 and large capacity of pretrained CNNs shown in Table 3 shows that our suggested technique is appropriate for the data's complexity. Similarly, our proposed BreastUNet model outperforms most existing conventional CNN designs, owing to the differences in properties between histopathological images. The proposed technique's training step is significantly time-consuming, since it comprises two training phases and individual optimization tuning. The test phase, on the other hand, moves quickly. The training step of optimization is more computationally costly than that of pertinent feature extraction. The training of different pretrained CNNs is no longer an issue because to the availability of GPUs and cloud-based and open-source services. For complicated issues, the use of high-capacity sophisticated CNNs is appropriate. We plan to investigate new and novel ensemble learning strategies in the future. We will work on establishing an ensemble approach in which pre-trained CNNs are used to include feature spaces from several auxiliary learners during run-time, and a final decision is generated in an end-to-end way.

V. CONCLUSION
The mitosis count is critical in determining the pace of tumour growth and the grading system for breast cancer tumours. Mitotic nuclei are difficult to identify due to their heterogeneous composition and strong resemblance to dense nuclei and other cellular components. We present the heteromorphous ''BreastUNet,'' a deep CNN based heteromorphous hybrid that successfully distinguishes mitotic nuclei from non-mitotic nuclei. Four distinct CNN-based models are designed to capture the variance in the structure, texture, and morphological features of mitotic nuclei in this respect. The diversity and accuracy of the pretrained CNN's performance on the test set (range from 0.75-0.85) shows sufficient diversity and accuracy. The proposed heteremorphous BreastUNet surpasses state-of-the-art CNNs with a 7 percent to 9 percent improvement and current approaches in terms of F1 score (F1 score 0.95 and accuracy 0.95). This shows that a hybrid feature space made up of a variety of feature sets created by various CNN architectural designs can efficiently deal with the dataset's heterogeneous representation and imbalance. This approach will be expanded to WSIs in the future to automate the histopathology procedure from lab to final judgment.