A Comprehensive Review for Breast Histopathology Image Analysis Using Classical and Deep Neural Networks

Breast cancer is one of the most common and deadliest cancers among women. Since histopathological images contain sufficient phenotypic information, they play an indispensable role in the diagnosis and treatment of breast cancers. To improve the accuracy and objectivity of Breast Histopathological Image Analysis (BHIA), Artificial Neural Network (ANN) approaches are widely used in the segmentation and classification tasks of breast histopathological images. In this review, we present a comprehensive overview of the BHIA techniques based on ANNs. First of all, we categorize the BHIA systems into classical and deep neural networks for in-depth investigation. Then, the relevant studies based on BHIA systems are presented. After that, we analyze the existing models to discover the most suitable algorithms. Finally, publicly accessible datasets, along with their download links, are provided for the convenience of future researchers.


I. INTRODUCTION
B REAST cancer is the most commonly diagnosed and leading cause of cancer deaths among women [1]. According to the World Health Organization (WHO), every year 2.1 million women have breast cancer worldwide. In 2018, an estimated 627,000 women died, representing about 15% of all cancer deaths among women [2]. In the United States, it ranks first in the record of the most common cancers that women are expected to be diagnosed in 2019 at a rate of up to 30% [3].
There are four types of breast tissue i.e., normal, benign, in-situ carcinoma, and invasive carcinoma. Benign tissue refers to a minor change in the structure of the breast, but it is not classified as cancer, and in most cases, it is not harmful to health. In-situ carcinoma remains in the mammary duct lobule system and does not affect other organs. If it diagnoses in time, in-situ carcinoma can be cured. However, invasive carcinoma is a malignant tumor that tends to spread in other or-gans. There are many techniques for breast cancer detection, such as X-ray mammography [4], 3-D Ultrasound (US) [5], Computed Tomography (CT), Positron Emission Tomography (PET) [6], Magnetic Resonance Imaging (MRI) [7], and breast temperature measurement [8]. However, pathological diagnosis is often regarded as a "golden standard" [9]. For better observation and analysis, the removed tissues usually need to be stained, where the Hematoxylin and Eosin (H&E) staining approach is the most common method. The hematoxylin dyes the nuclei a dark purple color and the eosin dyes other structures (cytoplasm, stroma, etc.) a pink color. Make it like FIGURE 1, showing the different types of breast tissue images stained with H&E.
In histopathological research, the sections are examined under a microscope to analyze the characteristics and properties of the tissues by a histopathologist [11]. Traditionally, the tissue sections are observed by the naked eyes of the histopathologist directly, and the visual information is ana-  [10].
lyzed based on the prior medical knowledge manually. However, due to the complexity and diversity of histopathological images, this manual analysis can take much time. At the same time, the objectivity of this manual analyzing process is unstable, depending on the experience, workload, and mood of the histopathologist greatly. In recent years, Artificial Intelligence (AI) technology develops rapidly. In particular, it makes important achievements in computer vision, image processing and analysis. AI also shows potential advantages in histopathological analysis. AIassisted diagnosis can undertake tedious focus screening work and quickly extract valuable information related to diagnosis from massive data. Meanwhile, AI has a strong objective analysis ability in histopathological detection and can avoid subjective differences caused by manual analysis. To some extent, the misjudgment of pathologists can be reduced and the working efficiency can be improved.

A. GENERAL DEVELOPMENT OF EXISTING AI ANALYSIS HISTOPATHOLOGY
AI is an umbrella term encompassing the techniques for a machine to mimic or go beyond human intelligence, mainly in cognitive capabilities [12]. As shown in FIGURE 2, the main contents of AI research include Machine Learning (ML), pattern recognition, natural language processing, etc. Especially, ML has a significant contribution to the development of medicine.
ML is used in the pathological diagnosis of different cancer fields, e.g., cervical cancer, gastric cancer, colon cancer, lung cancer and breast cancer. The scope of the application focuses on the benign and malignant diagnosis, disease grading, staining analysis, and early tumor screening. For example, the work of [13] proposes a weakly supervised multi-layer hidden conditional random field model to classify the cervical histopathological images into well, moderate and poorly differentiated stages. In the experiment, the proposed method is tested on the six cervical IHC datasets and obtains an overall classification accuracy of 77.32% and the highest one of the six is 88%, showing the effectiveness and potential of the method. In the field of gastric cancer, the work of [14] proposes a deep learning based framework, namely GastricNet, for automatic gastric cancer identification. The experimental results show that this deep learning framework performs better than state-of-the-art networks like DenseNet, ResNet, and achieves an accuracy of 100% for slice-based classification. The colorectal cancer is a malignant tumor that starts in the form of growth known as polyps mainly in the inner linings of the colon or rectum part. In the work of [15], proposing an automated supervised technique using deep learning to keep original image size is proposed in this paper for doing five-grade cancer classification via 31 layers Deep Convolutional Neural Network (DCNN). The proposed model results in classification accuracy of 96.97% for two-class grading and 93.24% for five-class cancer grading. About lung cancer pathology, Adenocarcinoma (LUAD) and squamous cell carcinoma (LUSC) are the most prevalent subtypes of lung cancer. The study of [16] trains a DCNN (inception-v3) on Whole Slide Images (WSIs) obtained from The Cancer Genome Atlas to accurately and automatically classify them into LUAD, LUSC or normal lung tissue, with an average Area Under the Curve (AUC) of 0.9.
It is worth noting that ANNs, as a branch of machine learning, play an important role in pathological diagnosis. ANN method, including classical and deep neural networks, is a kind of mathematical model or calculation model which imitates the structure and function of the biological neural network. In recent years, ANNs are widely used in Breast Histopathological Image Analysis (BHIA) for image segmentation, feature extraction, and classification. The development trend of BHIA using ANNs is shown in FIGURE3.

B. MOTIVATION OF OUR REVIEW PAPER
As far as we know, there exist some survey papers that summarize papers related to the BHIA work. (e.g., the reviews in [9], [12], [17]- [34]) In the following part, we go through the survey papers that are related to the BHIA work.
The survey of [9] reviews machine learning methods that are usually employed in histopathological image processing, such as segmentation, feature extraction, unsupervised learning, and supervised learning. More than 130 papers about histopathological image analysis are summarized, but only five are about BHIA with ANNs.
The survey of [12] publishes a research survey, focusing on the use of AI and deep learning in the diagnosis of breast pathology images, and other recent developments in digital image analysis. Among them, we are only interested to summarize the development of deep learning in breast pathological diagnosis from the application direction. However,   the article does not discuss the results that have been obtained for each research method. The survey of [30] summarizes current deep learning techniques in mammography and breast histology. In this article we only focus on deep learning techniques on breast histopathology images. According to different tasks, namely nuclei analysis, tubular analysis, epithelial and stromal region analysis, mitotic activity analysis and other tasks in the breast digital histopathology image process, 16 papers are summarized based on BHIA.
The survey of [31] summarizes the deep learning applications in breast cancer image analysis in Screen-File Mammography (SFM), Digital Mammography (DM), US, MRI, and Digital fast tomosynthesis (DBT) imaging modes, respectively. At the same time, six papers are found based on the topic of our interest.
In [32], an overview of "recent trends in computer assisted diagnosis system for breast cancer diagnosis using histopathological images" with 106 related works is presented. This review summarizes those works by four technical steps, including image pre-processing, segmentation, feature extraction and selection, as well as classification. However, there are only 20 related works about BHIA with ANNs in this paper.
The survey of [33] publishes a review about the classification tasks of breast cancer on deep learning. The author reviews five aspects, namely datasets used, various medical imaging modalities exploited, image pre-processing techniques, types of DNNs, and the performance metrics used to construct and evaluate breast cancer classification models. The paper cites 49 studies, of which 27 are about histopathological images, and the rest are about mammograms. In the empirical evaluation, the analysis based on histopathological images and mammograms is not distinguished, and the time is only 2014-2018.
In our previous work [34], we propose a brief survey for breast histopathology image analysis using classical and deep neural networks. With more than 60 related works, referring to classical ANNs, deep ANNs and methodology analysis.
Besides our previous brief review about BHIA with ANN techniques, there is not a special one that focuses on the ANN approaches in this field. Hence, in order to clarify the BHIA work using ANN approaches in recent years, as of early 2020, based on our previous work in [34], we summarize more than 150 related works to prepare this comprehensive review. We propose this paper with the following structure: In Sec. II, the BHIA work using classical ANN methods are introduced; in Sec. III, the state-of-the-art deep ANN methods are summarized; Sec. IV, presents the method analysis; Sec. V concludes this paper and discusses the future work. VOLUME 4, 2016 II. BHIA USING CLASSICAL ANNS An overview of the BHIA work using classical ANN methods is compiled in this section. Then, we analyze and summarize the chapter.

A. RELATED WORKS
In this section, we divide related work into classification tasks and segmentation tasks according to the motivation. Then, we summarize the contributions, methods, and results of each paper.
a: Classification Tasks: In [35], in order to evaluate two proposed texture features, a third-party software (LNKnet package) containing a neural network classifier is used. In the experiment, 536 samples are used for classifier training and 526 samples are used for testing. Finally, an accuracy of 90% is achieved.
In [36], Support Vector Machine (SVM), k-Nearest Neighbor (KNN) and Probabilistic Neural Networks (PNN) classifiers are combined with signal-to-noise ratio feature ranking, sequential forward selection-based feature selection and principal component analysis feature extraction to distinguish the benign and malignant tumors of the breast. Finally, the best overall accuracies for breast cancer diagnosis are achieved by using an SVM classifier. The accuracy of 98.80% is achieved on dataset 1 (692 specimens of fine-needle aspirates of breast lumps), and 96.33% is achieved on dataset 2 (295 microarrays). Similarly, PNN achieves 97.23% and 93.39% overall accuracy on dataset 1 and dataset 2, respectively.
In [37], four types of H&E stained breast histopathology images are classified, using eight features and a three-layer forward/back ANN classifier. In the experiment, 1808 training samples, 387 validation samples, and 387 test samples are tested, and an overall accuracy around 95% is achieved.
In [38]- [40], an automatic breast cancer classification scheme based on histopathological images is proposed. First, edge, texture and intensity features are extracted. Then, based on each of the extracted features, an ANN classifier is designed, respectively. Thirdly, an ensemble learning approach, namely "random subspace ensemble", is used to select and aggregate these classifiers for even better classification performance. Finally, a classification accuracy of 95.22% is obtained on a public image dataset.
In [41], in order to classify low magnification (10×) breast cancer histopathology images (H&E stained) into three malignancy grades, 30 texture features are extracted first. Then, feature selection is applied to find more effective information from the extracted features. Thirdly, a PNN classifier is built up based on the selected features. Lastly, 65 images are tested in the experiment, and an overall accuracy around 87% is obtained.
In [42], morphological features are extracted to realize the classification of cancerous and non-cancerous cells in histopathological images, and 70 histopathological images are randomly selected as the dataset. In the experiment, a multi-layer perceptron, based on feed-forward artificial neural network modal, achieves 80% accuracy, 82.9% sensitivity and 89.2% AUC, respectively.

b: Segmentation Tasks:
In [43], a competitive neural network is applied as a clustering based method to segment breast cancer regions from needle biopsy microscopic images. In this work, 21 shape, texture and topological features are extracted. Then, the network is used to cluster the images into different regions based on these features. In the experiment, a dataset with over 500 images is tested, and an overall accuracy of around 98.7% is achieved.
In [44], a supervised segmentation scheme using multilayer neural network and color active contour model to detect breast cancer nuclei is proposed. In this work, 24 images are used to test the method, and an average accuracy of 95.5% is finally achieved. The flow chart is shown in FIGURE 4.

B. SUMMARY
According to the review above, we can see that the ANNs used in the BHIA field around 2012 are classical neural networks. The classical neural network has remarkable performance in various fields, but it also has some limitations, such as easier to overfit, slow training speed, and can only set parameters according to experience. Due to the low computational speed of the computer and the lack of sufficient data to train the computer system at the time, it is impossible to extract the effective ANN features from the raw data. Therefore, most of the classical neural networks in the field of BHIA are used as classifiers. In the aspect of feature selection, most research works use texture features and morphological features for segmentation and classification. TABLE 1 summarizes the work of different teams using classical neural networks in analyzing histopathological images of breast cancer. Further details of the method analysis are discussed in Sec. IV-A

III. BHIA USING DEEP NEURAL NETWORKS
In the analysis of breast histopathology images based on deep neural networks, some publicly available datasets are frequently applied. As shown in TABLE 2, we provide detailed information and download links for the datasets mentioned in our review.

A. RELATED WORKS
In this section, we group related work according to the applied datasets. Then, we summarize the motivation, contribution, methods, and results of each paper in chronological order. In 2015, BreaKHis dataset was released in [52]. This dataset is composed of 7,909 histopathological images from 82 clinical breast cancer patients. All the histopathological images of breast cancer are three channel RGB micrographs with a size of 700× 460 pixels. Since objective lenses of different multiples are used in collecting these histopathological images of breast cancer, the entire dataset comprises four different subdatasets, namely 40×, 100×, 200×, and 400×. All of these sub-datasets are grouped into benign and malignant tumors. Based on this dataset, many related works are carried out. VOLUME 4, 2016 Related Works of BreaKHis in 2016: In [53], the classification of breast cancer histopathological images by a Convolutional Neural Network (CNN) independent of magnification is proposed. This paper uses two different architectures: Single-task CNN is used to predict malignancy, while multi-task CNN is used to predict both malignancy and image magnification levels simultaneously. FIGURE 5 is the overall process of this work. Finally, the average recognition rate of the single-task CNN model in the benign/malignant classification task is 83.25%. The average recognition rate of the multi-task CNN model in the benign/malignant classification task is 82.13% and the average recognition rate in the magnification estimation task is 80.10%.

Related Works of BreaKHis in 2017:
In [54]- [56], based on LeNet and AlexNet, deep ANN methods are used to classify breast histopathology images in the BreaKHis dataset. In the experiment, the dataset is divided into training (70%) and testing (30%) sets, and an overall accuracy around 85% is obtained.
In [57], a transfer learning work is carried out, where an image is first represented by Fisher Vector (FV) encoding of local features extracted using the CNN model pre-trained on ImageNet. Then, a new adaptation layer is designed to finetune the whole deep learning structure. Finally, an accuracy around 87% is achieved on 30% testing images. Similarly, in [58], another transfer learning strategy is applied to the same task, and achieves an overall accuracy around 90%.
In [59], a deep learning structure with a single convolutional layer is proposed for classification task, which obtains an accuracy of 77.5%. In contrast, in [60], a deep learning model with multi-layer CNNs is built up, and obtains an accuracy up to 90%. Furthermore, in [61], a CNN model, namely the "Class Structure-based Deep CNN" (CSDCNN), is proposed to represent the spatial information within a deep CNN.
In [62], a DCNN based whole-slide histopathology classifier is presented. First, the posterior estimate of each view at a specific magnification is obtained from CNN at a specific magnification. Then the posterior estimate across random multi-views at multi-magnification is voting filtered to provide a slide level diagnosis. Finally, the experiment uses a patient-level 5-folded cross-validation and achieves an average accuracy of 94.67%, sensitivity of 96%, specificity of 92% and F-score of 96.24%.
In [63], a new method for breast cancer histopathological image classification based on DCNNs is proposed, called the BiCNN model, for two-class classification problems on pathological images. The BiCNN has more depth, more width and more complex architecture, which has little parameters and reliable performance. In the experiment, the average recognition rate for patient level is achieved as 97%.

Related Works of BreaKHis in 2018:
In [64], different ResNet structures are tested and compared for this task. The ResNet-V1-152 model obtains the best performance with an overall accuracy of 99.6% after 3000 epochs. Similarly, in [65], the effectiveness of three well recognized pre-trained transfer learning models (VGG-16, VGG-19, and ResNet-50 networks) are compared in this task. In the experiment, the VGG-16 with a logistic regression classifier obtains the best performance of 92.6% accuracy. Furthermore, in [66], Inception-V1, Inception-V2, and ResNet-V1-50 based transfer learning methods are compared, and ResNet-V1-50 obtains the highest accuracy of 95%.
In [67], two restricted Boltzmann machine and back propagation based DCNN models are proposed. Using these two models, the best average accuracy of 87.75% is obtained. Furthermore, in [68], based on CNN and Recurrent Neural Network (RNN) algorithms, a combined deep learning structure is introduced. In this work, unsupervised learning algorithms are first used to segment different tissues into different regions. Then, based on the segmentation result, the proposed deep learning approach is applied to the final classification task. Lastly, an accuracy of 91% is achieved. In addition, in the work of [69], five DCNN models are built up, considering handcraft features and deep learning features jointly. In the experiment, the second model obtains the best performance of 92.19% accuracy.
In [70], a classification approach via deep active learning and confidence boosting is introduced, and achieves an overall accuracy of around 90%. Similarly, in [71], an implemented in-house CNN model is proposed, which combines the advantages of both machine learning features and classical color features.
In [72], a DenseNet based CNN model is proposed for classification, including four dense blocks and three transition layers. In the experiment, a 95.4% accuracy is achieved. Similarly, in [73], a ResNet based 152 layer deep learning model is built and achieves a correct classification rate of 98.77%.
In [74], CNNs are directly compared to classification based on hand-crafted features in binary classification (benign and malignant) and multi-class classification (benign and malignant sub-classes ) of breast cancer histological images. The results show that CNNs outperformed the hand-crafted feature based classifier, where the accuracy reach between 96.15% to 98.33% for the binary classification and 83.31% to 88.23% for the multi-class classification.
In [75], a multiple instance learning framework for CNN is proposed. A new pooling layer is proposed that would help to gather most of the informative features from the patches that make up the whole slide, without inter-patch overlap or global slide coverage. In the experiment, at 40×, 100×, 200× and 400× magnifications, the accuracy is 89.52%, 89.06%, 88.84% and 87.67%, respectively.
In [76], a new model for automatic classification of breast cancer tissue in histological images by DCNN is proposed, which does not consider the magnification factor of the image. The experimental results on BreaKHis achieved an average accuracy of 85.3%.

Related Works of BreaKHis in 2019:
In [78], a novel framework based on the hybrid attention mechanism is proposed to classify breast cancer histopathology images. This framework could automatically find useful regions from raw images, and thus does not have to resize the raw images for the network to prevent information loss. At four different magnifications, the average accuracy is about 96% while only 15% of raw pixels are used.
In [79], a transfer learning and supervised classifier based prediction model for breast cancer is proposed. As can be seen from FIGURE 6, four pre-trained ConvNets are used for transfer learning to extract image features, and then PCA is applied to the feature vectors to reduce feature dimension. Finally, SVM, KNN, and Logistic regression are respectively used to classify images. In the experiment, the best results are obtained on 40×. The ResNet-50 with SVM classifier has the maximum accuracy of 96.24% and the best recall value of 100%. On the other hand, with Inception ResNet-V2, SVM gives the highest precision of 96.55%.
In [80], Inception-V3 and Inception-ResNet-V2 are trained using transfer learning techniques for binary and multi-class classification of breast cancer histopathological images. The results show that the Inception-ResNet-V2 net- work achieves the best results at 40× magnification: In the binary classification task, the image level accuracy is 97.90%, and in the multi-classification task, the image level accuracy is 92.07%.
In [81], a breast cancer histopathology image classification network (BHCNet) is designed. BHCNet includes one plain convolutional layer, three SE-ResNet [82] blocks, and one fully connected layer. Each SE-ResNet block is stacked by N small SE-ResNet modules, which is denoted as BHCNet-N . In the results, the BHCNet-3 achieves the accuracy between 98.87% and 99.34% for the binary classification and the BHCNet-6 achieves the accuracy between 90.66% and 93.81% for the multi-class classification.
In [83], deep learning, transfer learning and Generative Antagonistic Network (GAN) are combined to improve the accuracy of breast cancer classification on a limited training dataset. First, the fine-tuned VGG-16 and VGG-19 are used to extract features and sent to CNN for classification. In addition, StyleGAN [84] and Pix2Pix [85], two GAN models, are applied to generate 4,800 and 2,912 fake images, respectively. In the experiment, the proposed method is evaluated on the BreaKHis dataset and two generated datasets from BreaKHis by GAN. The experiments show that GAN images created much noise and affected classification accuracy. Finally, the best result is obtained in BreaKHis dataset. The accuracy is 98.1% in the binary classification.
In [86], a method of classifying breast cancer histopathologic images based on double transfer learning is proposed. This method can be divided into two steps. In the first step, as shown in FIGURE 7, in order to improve the quality of the dataset before training the classifier using the BreaKHis dataset, an SVM is trained to classify relevant and irrelevant images in histopathological images, and then it is used as a filter to eliminate irrelevant images from the BreaKHis dataset. In the second step, as shown in FIGURE 8, another SVM is trained to classify benign and malignant. Both steps use transfer learning (Inception-v3 CNN pre-trained with ImageNet dataset). The best classification accuracy can be obtained by Inception-v3 + filter (Inception-v3 is used to extract features, and filtering refers to the removal of irrelevant images) method at 100× and 200× magnifications of 91% and 89%, respectively.

Related Works of BreaKHis in 2020:
In [87], a novel feature extraction method is proposed for the classification of breast histopathological images. First, the images are divided into small pieces that are not overlapped. Then, pre-trained CNNs (10 models in total) are used for feature extraction. Finally, an SVM is applied as a classifier. In the experiment, the best patient-level accuracy is obtained by the AlexNet-SVM model. At 40×, 100×, 200× and 400× magnifications, the accuracy is 89.46%, 92.61%, 93.92%, and 89.78%, respectively.
In [88], a ResHist model is designed, which is a residual learning-based 152-layered CNN to classify the histopathological images of breast cancer. In the experiment, histopathological images are first augmented and the ResHist model is trained end-to-end on the augmented dataset in a supervised learning manner. Finally, images are classified into benign and malignant categories by using the trained ResHist model. The result shows that the ResHist model achieves a best accuracy of 92.52% and a F1-score of 93.45%. In addition, in order to study the discrimination ability of deep features of ResHist model, the extracted feature vectors are fed into KNN, random forest [89], quadratic discriminant analysis, and SVM classifiers. Among them, when the deep features are fed back to the SVM classifier, the best accuracy of 92.46% is achieved. b: "Camelyon" Tasks: "Camelyon Grand Challenge" is a task to evaluate computational systems for the automated detection of metastatic breast cancer in WSIs of sentinel lymph node biopsies.
Related Works of Camelyon 2016: In [90], a DCNN is built for this task, and achieves an AUC of 97%. In [91], a GoogLeNet based deep learning method is introduced, where 270 images are used for training, and 130 are used for testing. Lastly, an AUC of 92.5% is obtained. With the same experimental setting, in [92], a recurrent visual attention model is proposed, which includes three primary components composed of dense or convolutional layers to describe the information flow between components within one time-step. Finally, a 96% AUC is achieved.
In [93], a fast and dense screening framework (ScanNet) for detecting metastatic breast cancer from WSIs is proposed. ScanNet is implemented based on the VGG-16 network by changing the last three fully connected layers to fully convolutional layers. In the result, Free Response Operating Characteristic (FROC) of 0.8533 and AUC of 98.75% are obtained.
In [94], Multiple Magnification Feature Embedding (MMFE) is introduced, which is an approach using transfer learning to detect breast cancer from digital pathology images without network training. The main idea of the MMFE method is to simulate the daily diagnosis process of a medical doctor. First, a low-resolution image is observed to identify suspicious areas. Then, it is switched to a high-resolution image for further confirmation. Experiments show that this approach can greatly improve the training and prediction speed of the model without reducing the performance of the model.
In [95], a summary of the Camelyon 2016 shows that: 25 of 32 submitted algorithms are deep learning based methods, and 19 top-performing algorithms are all DCNN approaches.
Related Works of Camelyon 2017: In the Camelyon 2017 [96], in order to detect four types of breast cancer from the WSIs, a deep learning architecture is proposed with limited computational resources. In this work, two CNNs are applied in a cascade, followed by local maxima extraction and SVM classification of local maxima regions. In the experiment, 300 images are used for training, 200 are used for validation, 500 are for test, and an accuracy of 92% is finally achieved. The Grand Challenge on BreAst Cancer Histology images (BACH) is co-organized with the 15th International Conference on Image Analysis and Recognition (ICIAR 2018) [10]. There are two goals in this challenge. Part A of the challenge consists of automatically classifying H&E stained breast histology microscopy images in four classes: Normal, benign, in-situ carcinoma and invasive carcinoma. The data in part A is composed of 400 H&E stained breast histology images.
All images are of equal dimensions (2048×1536 pixels). Part B consists of performing pixel-wise labeling of WSIs in the same four classes as Part A. The data in part B is composed of 20 WSIs of very large size. Each WSI can have multiple normal, benign, in-situ carcinoma and invasive carcinoma regions.
In order to classify four types of breast cancer histopathology images, an Inception-V3 based deep learning model is introduced in [97]. In the experiment, 300 images are used for training, and 100 are used for testing. Finally, an average accuracy of 85% is achieved.
A two-stage CNN model is also proposed in [98], where the first stage is for pixel-level classification, and the second stage is for image-level classification. In the experiment, an overall accuracy around 94% is obtained. Similarly, in [99], a two-stage classification approach is proposed. In the first stage, an AlexNet based feature extraction is applied. In the second stage, three different classifiers are used. In the experiment, an SVM classifier achieves the best result (99.84% accuracy). Similarly, in [100], the AlexNet is also applied as a basic model to build a hierarchical classification model, and an accuracy of 95% is obtained.
In [101], a transfer learning-based approach for the classification of breast cancer histology images is presented. Inception-V3 and ResNet-50 CNNs, both pre-trained on the ImageNet database, are used. In the experiment, the Inception-V3 network achieves an average test accuracy of 97.08% for four classes, marginally outperforming the ResNet-50 network, which achieves an average accuracy of 96.66%.
In [102], an Inception ResNet-V2 is proposed to classify histological images of breast cancer through transfer learning, fine-tuning, and data augmentation. Out of 100 images in each class, 70, 20, and 10 images are randomly selected for training, testing, and validation. The final results show that the accuracy of the test set is 90% and the loss is 0.59, while the accuracy of the validation set is 93% and the loss is 0.23.
In [103], a deep learning approach for analyzing breast histology images at both micro-level (patch-based image classification) and macro-level (WSI segmentation) is proposed. The approach contains two networks, as shown in FIGURE 9, which share the architecture and weights, especially the encoder in the network, to improve the utility of the trained network and available dataset. In the result, for patch classification, 71% and 65% accuracy are obtained on the training and test datasets, respectively. In terms of segmentation, an overall score of 0.7343 and 0.4945 are obtained on the training and test data sets, respectively.
In [104], approaches for the classification of microscopic images as well as the segmentation of WSIs are presented. In both parts of the challenge, data preparation, scale selection, and augmentation are firstly carried out. In part B of the challenge, additional data are added. Finally, network training is conducted. The densenet-161 architecture with pretraining on ImageNet is selected and repeatedly trained in FIGURE 9: Overall structure of the proposed model in [103]. A classification network consists of an encoder and two processing layers. A segmentation network contains an encoder and decoder. This figure corresponds to Fig.1 in the original paper.
the expansion training set. The results show that a 96.24% accuracy is achieved.
In [105], a method for classification of breast cancer histopathology images based on deep learning is proposed. The effects of various preprocessing methods are compared, and the classification results of CNN and CNN with SVM are also compared. Finally, an accuracy of 83% is obtained in part A task.
In [106], a context-aware network for automated classification of breast cancer histopathological images is proposed. The method mainly includes two steps: First, the activation feature of a trained ResNet is used to classify the non-overlapping patches. Then, an SVM classifier is trained to classify the patches of overlapping blocks. Finally, the majority-voting method is used for image-wise classification. As a result, an average accuracy of 83% is obtained.
In [107], six different feature extractors are compared: Hand-crafted features, ResNet-18, ResNeXt, NASNet-A, ResNet-152 and VGG-16. The result shows that the pretrained deep learning network on ImageNet has better performance than the popular hand-crafted features used for breast cancer histology images. Finally, the integration method based on random forest dissimilarity is used to combine hand-crafted features with five deep learning feature groups, and an average accuracy of 87.1% is obtained.
In [108], a deep learning framework for multi-class breast cancer image classification is presented. The framework of the approach is illustrated in FIGURE 10. First, Inception-V3 is used for patch-wise classification. Then, the patchwise predictions are passed through an ensemble fusion framework involving majority voting, a Gradient Boosting Machine (GBM), and logistic regression to obtain the imagewise prediction. Finally, an average accuracy of 87.5% is obtained. VOLUME 4, 2016 FIGURE 10: An overview of the proposed framework in [108]. This figure corresponds to Fig.2 in the original paper.
In [109], a new hybrid convolutional and recurrent deep neural network for breast cancer histopathological image classification is proposed. First, a fine-tuned Inception-V3 is used for feature extraction for each image patch. Then, the feature vectors are input into a 4-layer Bidirectional Long Short-Term Memory network (BLSTM) [110] for feature fusion. Finally, a complete image-wise classification is carried out. In the experiment, an average accuracy of 91.3% is obtained in image-wise. Notably, a new dataset containing 3,771 histopathological images of breast cancer is published in this paper. It covers as many different subclasses spanning different age groups as possible. The dataset is publicly available at [111].
In [112], a patch-based classifier (PBC) using CNN for automatic classification of histopathological breast images is proposed. The proposed classification system works in two different modes: One patch in one decision (OPOD) and all patches in one decision (APOD). The flowchart of OPOD technology for patch classification is shown in FIGURE 11. OPOD is mainly responsible for predicting the class labels of each patch extracted from the pre-processed histopathological images. As shown in FIGURE 12, APOD technology classifies images by majority voting for each patch predicted by OPOD technology. In the test set, APOD technology achieves accuracy of 90% in 4-class classification and 92.5% in 2-class classification.
In [113], a method for the diagnosis of breast cancer histopathology images based on transfer learning and global pooling is proposed. Five DCNN architectures are used as feature extractors, namely Inception-V3, InceptionResNet-V2, Xception, VGG-16, and VGG-19. The experimental results show that the network structure based on the pre-trained Xception model is better than all other DCNN structures in average classification accuracy, reaching 92.50%.  mitotic cells on images of both scanners and 322 mitotic cells on the multispectral microscope [114].
In [115], [116], manually designed color, texture, and shape features are jointly used with the machine learning features extracted by a multi-layer CNN. Finally, this method obtains an F1-scores up to 65.9% on color scanners and 58.9% on multi-spectral scanners. Similarly, in the work of [117], handcrafted features and DCNN features are used in an ensemble learning process together, and an F1-score of 73.5% is obtained.
In [118], in order to detect the mitosis in a breast histology image, a deep max-pooling CNN is built up, which is trained to classify each pixel in the image into a labeled region. In the experiment, 26 images are used for training, 9 for validation, and 15 for test. Finally, an F1-score of 78.2% is achieved. Furthermore, a similar method is used in the work of [119], and an F1-score of 61.1% is obtained.
In [120], a novel deep cascade convolutional neural net-work (CasCNN) is designed to detect mitosis. CasCNN consists of two parts. First, using full CNN, a rough retrieval model is proposed to identify and locate mitotic candidates while maintaining high sensitivity. Then, a fine recognition model based on cross-domain knowledge transfer is proposed to further single out mitoses from the rough model. In the experiment, both the ICPR12 dataset and ICPR14 dataset are used. On the ICPR12 dataset, the precision of 80.4%, recall of 72.2% and F1-score of 78.8% are obtained. On the ICPR14 dataset, the precision of 46%, recall of 50.7% and F1-score of 48.2% are obtained. In [114], a summary of the ICPR 2012 contest shows that 17 teams submit their results and the IDSIA team gets the best performance. In the work of the IDSIA team, a CNN is trained through ground truth mitosis provided in training dataset, and then the CNN is used to calculate a map of mitosis probabilities on the whole image. Finally, achieving a recall of 70%, the accuracy of 89%, and the F-measure of 78%. e: "TCUG16" Tasks: The Tumor Proliferation Assessment Challenge 2016 (TU-PAC16) is the first challenge to predict tumor proliferation scores from WSIs. This challenge is organized in the context of the MICCAI 2016 conference in Athens, Greece. The goal of the challenge is to assess algorithms that predict the tumor proliferation scores from the WSIs. There are two tasks in this challenge. Task 1 is to predict the proliferation score based on mitosis counting. Task 2 is about the prediction of proliferation score based on molecular data. The participants can submit their results for either or both of the tasks.
In [121], a transfer learning system based DCNN algorithm is suggested for the segmentation and detection of mitoses in breast cancer histopathological images. This system uses two CNNs. A pre-trained CNN is used for segmentation, and Hybrid-CNN is employed for mitotic classification. Finally, in the task of mitosis detection, an F-measure of 71.3% with 76% area under the precision-recall curve is achieved.
In [122], a novel technique for the detection of mitosis by virtue of semantic segmentation is presented, called SegMitos. At the same time, a novel concentric label and concentric loss are proposed, which can train a dense prediction model with weak annotation. The idea of the experiment is as follows. First, the preparation of data and to produce concentric labels. Then to train the SegMitos model. Finally, the trained model is deployed to the testing images of the mitosis dataset. As a result, four datasets (ie: 2012 ICPR MITOSIS dataset, MITOS-ATYPIA-14 dataset, AMIDA13 dataset, and TUPAC16 dataset) are utilized to validate the proposed method. Finally, on the TUPAC16 dataset, an Fscore of 66.9% is obtained.
In [123], a summary of the TCUG16 shows that: 12 teams submit results for the first task, and 6 teams submit results for the second task. With the exception of one team, all teams use DCNNs as part of the processing pipeline. For the first task, the best performing method achieves a quadratic-weighted Cohen's kappa score of k = 0.567, 95% CI [0.464, 0.671] between the predicted scores and the ground truth. For the second task, the predictions of the top method have a Spearman's correlation coefficient of r = 0.617, 95% CI [0.581, 0.651] with the ground truth. In 2015, the Bioimaging 2015 Breast Histology Classification Challenge dataset was released in [124]. This dataset is composed of high-resolution (2048 × 1536 pixels) and H&E stained breast cancer histology images. All the images are digitized with the same acquisition conditions, with the magnification of 200× and pixel size of 0.42µm × 0.42µm. There are four different types of histological images of breast cancer in the dataset, namely normal tissue, benign lesion, in-situ carcinoma and invasive carcinoma. A total of 285 images are included in the dataset. Of these, 249 images are used for the training set and 36 images are used for the test set. The pictures in the test set are divided into two groups, namely the initial group (20 images with less classification difficulty) and the extended group (16 images with more difficult classification).
In [124], a DCNN model is introduced to classify four breast cancer histopathology types in the whole slide image. In the experiment, 249 images are used for training, 20 images are used for testing, and accuracy of 77.8% is obtained.
In [125], in order to classify different breast cancer types in H&E stained histopathology images, pre-trained ResNet-50 and ResNet-101 networks are applied with a fine-tuned process and a fusion strategy. In the experiment, the bioimaging 2015 breast histology classification challenge datasets and BACH datasets are tested. Finally, 97.22% and 88.5% accuracies are obtained on the Bioimaging 2015 Breast Histology Classification Challenge dataset and BACH dataset, respectively.
In [126], an extended version of the Bioimaging 2015 Breast Histology Classification Challenge dataset is used. This dataset is the same as the original data set in terms of image type, acquisition conditions, and size. The total number of images in this dataset is 400. The pre-trained ResNet-50, Inception-V3, and VGG-16 networks are fused into a deep learning structure and achieve a mean accuracy of 87.2%.
In [127], an approach based on deep learning for multiclassification of breast histological images is proposed. The framework of the approach is illustrated in FIGURE 13. Firstly, two patches of different sizes are extracted from the histological images of breast cancer by sliding window mechanism, including cell-level and tissue-level features. In order to solve the problem of insufficient diagnostic information or label information error in some sampling patches, a batch screening method based on CNN and k-means is proposed to select more discriminant patches. Then, ResNet-50 is used as the feature extractor to extract features from the patch and P-norm pooling is used to obtain the final image VOLUME 4, 2016 features. Finally, an SVM is used for the final image classification. The result shows that a 95% accuracy is achieved on the initial test set. FIGURE 13: A schematic illustration of the proposed framework in [127]. This figure corresponds to Fig.3 in the original paper.
In [128], a method for breast cancer classification with the Inception Recurrent Residual Convolutional Neural Network (IRRCNN) model is proposed. The IRRCNN approach is applied for breast cancer classification on two publicly available datasets including the Bioimaging 2015 Breast Histology Classification Challenge dataset and BreaKHis. On the Bioimaging 2015 Breast Histology Classification Challenge dataset, test accuracy of 99.05% and 98.59% is obtained for the binary and multi-class classifications, respectively. On the BreaKHis dataset, the accuracy of 97.95 ± 1.07% (image-level) and 97.65 ± 1.20% (patient-level) is obtained for the binary classification, respectively. In addition, the accuracy of 97.57 ± 0.89% (image-level) and 96.84 ± 1.13% (patient-level) is obtained for the multi-class classification, respectively.
In [129], transfer learning based on AlexNet, GoogleNet, and ResNet is used to classify the histopathological images of breast cancer. The result shows that ResNet has the highest accuracy, achieving 83.60% and 85.0% accuracy at the patch level and image level, respectively. g: "IDC" Tasks: Invasive Ductal Carcinoma (IDC) dataset is a publicly available dataset that was first introduced by Cruz-Roa et al. [130]. The dataset contains digital breast cancer histopathological slides from 162 women with IDC. All slides are digitized via a whole-slide scanner at 40× magnification. The dataset contains 277,524 patches of size 50×50 pixels (198,738 IDC negative and 78,786 IDC positive).
In [130], a deep learning approach for automatic detection and visual analysis of IDC tissue regions in WSIs of breast cancer is presented. The framework of the approach is il-lustrated in FIGURE 14. First, grid-sampling image patches of all the regions containing tissue in WSI. Then, CNN is trained from the sampled patch to predict the probability of patch belonging to IDC. Finally, a probability map is built on the WSI, highlighting patches that have IDC with a probability greater than 0.29. In the experiment, 162 original slices are divided into 3 different subsets: 84 of them are used for training, 29 are used as the validation set and 49 are used for testing. As a result, an F-measure of 71.80% and an accuracy of 84.23% are obtained for automatic detection of IDC regions in WSI.

FIGURE 14:
Overall framework for automated detection of IDC in WSI using CNN in [130]. This figure corresponds to Fig.1 in the original paper.
In [131], a new CNN-based model for identifying IDC cells in histopathological slides is proposed. This model, which is derived from the Inception architecture, proposes a multi-level batch normalization module between each convolution step. In the experiment, 94,543 patches are used for training, 31,514 for validation, and 151,465 for testing. Finally, a balanced accuracy of 89% and an F1-score of 90% are obtained. h: Other Tasks:

1) Classification
In [132], a Principal Component Analysis Network (PCANet) is introduced to classify Ductal Carcinoma In-Situ (DCIS) and Usual Ductal Hyperplasia (UDH) images. In this work, a dataset with 20 DCIS and 31 UDH images are tested, where 10,000 patches are randomly sampled from the training set to learn the models. Finally, an accuracy around 79% is achieved.
In [133], a classification method based on CNN is proposed for the WSIs of breast tissue. In this work, two CNNs are trained. CNN-I is used to classify the WSI into the epithelium, stroma, and fat. CNN-II operates on the stromal regions output by classification of CNN-I, and then classifies the stromal regions as normal stroma or cancer-associated stroma. The dataset contains 646 sections of breast tissue stained with H&E. In the experiment, 270 images are considered for training, 80 copies are for validation, and 296 are for testing. Finally, an area under Receiver-Operating Characteristic (ROC) of 0.921 is obtained.
In [134], to distinguish four breast cancer types in histopathological images, a deep learning method is introduced with hierarchical loss and global pooling. In this work, VGG-16 and VGG-19 networks are applied as the basic deep learning structures, and a dataset with 400 images are tested. In the experiment, 280 images are used for training, 60 images are for validation and 60 are for testing. Finally, an average accuracy of around 92% is obtained.
In [135], a work is carried out to classify five diagnostic breast cancer styles in the whole histopathological image. First, a saliency detector performs multi-scale localization of diagnostically relevant regions of interest in the images. Then, a CNN classifies image patches as five types of carcinoma. Lastly, the saliency and classification maps are fused for final categorization. In the experiment, 240 images are used to examine the effectiveness of the proposed method, and a 55% accuracy is finally achieved. The highlight of this work is that 45 pathologists take part in the final evaluation of the test images, and an average accuracy of around 65% is obtained. Hence, the performance of the proposed method is comparable to the performance of the pathologists that practice breast pathology in their daily routines.
In [136], an image analysis method is developed that uses deep learning to classify tumor grade, ER status, PAM50 intrinsic subtype, histological subtype, and recurrence risk score (ROR-PT). In the experiment, 571 examples of breast tumors are used for training and 288 are for testing. Finally, it can be distinguished from low-intermediate and high tumor grades (82% accuracy), ER status (84% accuracy), Base-like and non-base-like (77% accuracy), Ductal vs. lobules (94% accuracy), and high vs. low-medium ROR-PT score (75% accuracy).
In [137], pre-trained CNN architectures (GoogLeNet, VG-GNet, and ResNet) are used to extract features from images, and these features are fed to a fully connected layer. The average pool classification is used to classify malignant cells and benign cells. Two breast microscopic image datasets are used: The first is a standard benchmark dataset [52] and the other is developed locally at LRH hospital in Peshawar, Pakistan. In the experiment, 6000 images are considered to train the architecture and 2000 images are used for testing. Finally, an average classification accuracy of 97.53% is achieved.
In [138], Human epidermal growth factor receptor 2 (HER2) Scoring Contest is introduced. HER2 is an important prognostic factor for breast cancer, so the task of automatic HER2 scoring has great clinical significance. The paper shows that a total of 18 submissions from 14 teams are received for evaluation. Among the comprehensive results of all submitted automated methods, 8 of the top 10 teams used CNN-based learning methods. It can be seen that CNN-based learning methods play an important role in HER2 automatic scoring tasks.

2) Segmentation
In [139], a CNN based model with three hidden layers is built to segment the breast cancer cell nucleus in histopathological images. In this work, 58 H&E stained images are tested, and overall accuracy of around 95% is achieved on both RGB and Lab color spaces.
In [140], a fast scanning deep convolutional neural network (fCNN) is proposed for pixel-wise region segmentation. In the work, 92 images are used, which are selected from 20 patients in The Cancer Genome Atlas (TCGA) breast cancer dataset. 75 images are used for training and 17 images are used for testing. In the experiment, it takes only 2.3 seconds to segment an image with size 1000 × 1000 pixels. Also, a mean precision of 91%, a mean recall of 89%, and a mean F1-score of 85% are achieved.
In [141], a DCNN based feature learning is presented to automatically segment or classify epithelial and stromal regions in histopathological images. In this work, colorectal cancer dataset and breast cancer dataset are used separately. The breast cancer dataset consists of 157 H&E stained images. The data is acquired from two independent cohorts: Netherlands Cancer Institute (NKI: 106) and Vancouver General Hospital (VGH: 51). In the experiment, a superpixelbased scheme is used to over-segment the image into atomic regions. Then, the atomic regions are adjusted to square images of fixed size, and then they are fed back to the DCNN for feature learning. Finally, F-score of 85.21%, 89.10% and accuracy of 84.34%, 88.34% are obtained on NKI and VGH, respectively.
In [142], an automatic nuclei segmentation technique using DCNN is introduced. FIGURE 15 depicts a flowchart for the proposed framework. In the training stage, a DCNN model is trained and a kernel shape library based on a selection-based dictionary learning algorithm is obtained. In the test stage, the CNN model is applied to the images to generate probability maps, and then iterative region merging is performed to initialize the shape of each kernel. Then, the proposed kernel segmentation algorithm uses the local repulsive deformation model for shape deformation, and uses the shape priors of the sparse shape model for shape inference. Finally, an accuracy of 92% is achieved on the breast cancer dataset. In [143], an automated nuclei segmentation method is proposed. FIGURE 16 is the overall process of nuclei segmentation. This process can be divided into three main VOLUME 4, 2016 stages. First, the Sparse Reconstruction (SR) method is used to roughly remove the background and highlight the nuclei of the pathological image. Then the gradient descent technique is used to train the DCN cascade of the multi-layer convolutional network in order to effectively segment the nucleus from the background. At this stage, the patch and its corresponding label are randomly extracted from pathological images and input into the training network. Finally, morphological operation and prior knowledge are introduced to improve segmentation performance and reduce errors. In this work, the pixel segmentation accuracy of 92.45% and the F1-measure of 83.93% are obtained. In [144], a method of nuclear segmentation in histopathological images based on deep learning and mathematical morphology is proposed. In addition, an image dataset containing 33 images with 2754 annotated cells is provided. This dataset can be obtained at [145]. In this work, a set of manual annotation images is trained on a deep neural network and the posterior probability map is processed to achieve joint segmentation of the nuclei. Finally, the accuracy of 95.4%, recall of 77.3%, precision of 86.4% and F1-score of 80.5% are achieved.
In [146], an advanced supervised full CNN method for nuclear separation in histopathological images is proposed. First, a histopathological image is normalized to the same color space. Then a complete image is split into overlapping small blocks. The proposed nuclear boundary model is used to detect the nucleus and boundary on each plaque, and all the predictions are seamlessly recombined. Finally, fast and parameterless post-processing is applied to generate the kernel segmentation results. In the experiment, multiple organ H&E stained image dataset (MOD) [147], breast cancer histopathology image dataset (BCD) and breast cancer image dataset (BNS) [144] are used. Finally, an image with a size of 1000 × 1000 pixels can be segmented in less than 5 seconds.
In [148], an automatic end-to-end framework using deep neural networks for tissue-level segmentation is proposed. In this work, a new dataset of WSIs with different subtypes of breast cancer, consisting of 11 WSIs fully annotated, is tested. Finally, the results of U-Net, SegNet, FCN, and DeepLab are evaluated by using pixel-by-pixel indexes, with the Dice Coefficient (DC) values of 0.86, 0.87, 0.86, and 0.86, respectively.

3) Detection
In [149], a deep learning strategy named "Stacked Sparse Auto-Encoder" (SSAE), is presented to detect nuclei on high resolution breast cancer images.
In [150], a DCNN model is proposed to detect breast cancer metastasis in sentinel lymph nodes. In the experiment, 100 examples are used for training, 50 for validation, and 75 for testing. Finally, a sensitivity of 99.9% is achieved.
In [151], a novel accurate and high-throughput method (HASHI) for automatic invasive breast cancer detection in WSIs is presented. The test is conducted in three different data queues involving 500 cases. Finally, the comparison results of intensive sampling (6 million samples in 24 hours) and less samples (2000 samples in 1 minute) are obtained, an average DC reaches 76% on the independent test dataset.
In [152], handcrafted features are combined with highlevel features based on deep learning, which are directly fed back to the first fully connected layer for mitotic detection. In this work, three datasets are used, including the MITOS-ATYPIA-14 dataset, ICPR-2012 dataset, and AMIDA-13 dataset. Finally, a precision of 92%, a recall of 88% and an F-score of 90% are obtained.
In [153], a DCNN architecture is introduced to detect mitosis from histopathological images of breast cancer cells. The data set of MITOS atypia is tested. The result shows that the deeper CNN layer has better the performance of breast cancer image detection. In the 17 layer CNN architecture, 84.49% accuracy, 80.55% TPR, 11.66% FNR and 15.50% loss are achieved on average.

B. SUMMARY
From the survey above, we can find that deep ANN has been increasingly used in the field of BHIA since 2012. Among them, the method based on CNN is dominant. The main reasons for this development trend are as follows: (a) The emergence of high-performance GPU computing makes it possible to train networks with more layers. (b) More and more institutions have released datasets of breast histopathological images, to a certain extent alleviating the lack of labeled public datasets. The large increase in training data reduces the risk of over-fitting. (c) Compared with traditional image classification methods, deep learning can automatically learn features from data, avoiding the complexity and limitations of artificial design and feature extraction in traditional algorithms. (d) CNN has been widely applied in natural language processing, object recognition, image classification, and recognition, laying a foundation for the application of CNN to histopathological images of breast cancer. The work of different teams in the analysis of breast histopathological images using deep neural networks is summarized in TA-BLE 3. Further details of the method analysis are discussed in Sec. IV-B and Sec. IV-C.

IV. METHODOLOGY ANALYSIS
An overview of the deeper analysis of classical ANNs and deep ANNs is compiled in this section. Meanwhile, the outstanding methods in different tasks are analyzed.

A. ANALYSIS OF CLASSICAL ANN METHODS
According to the survey on classical ANNs, the Multi-Layer Perceptron (MLP) and Probabilistic Neural Network (PNN) are used more frequently in the analysis of breast histopathological images. Since the datasets employed in each work are different, the evaluation of each method cannot be evaluated longitudinally. Therefore, it is analyzed from the perspective of the neural network itself. MLP is known as a feed-forward neural network, which can solve the problem of linear inseparability and can be trained to accurately generalize when presented with new, unseen data [154]. However, the connection mode between its hidden layers is "fully connected", which causes too many training parameters. Therefore, It makes it difficult to have too many layers to solve more complex problems. At the same time, the learning speed of MLP is slow and it is easy to fall into local extremes. The papers involved in this article are [38], [39], [40], [42]. PNN is also a kind of feed-forward neural network. Compared with the MLP, the training speed of the PNN is faster and the PNN is usually more accurate than the MLP. The disadvantage is that it is slower than the MLP in classifying new cases and requires additional storage space to store the model. The papers involved in this article are [36], [41].
(e.g. [58], [66], [101], [121], [125], [129]) (2) Using a pretrained network as a feature extractor, and then using these features to train a new classifier. (e.g. [57], [65], [79], [83], [86], [87], [107], [109], [113], [126], [137]) In the transfer learning, the VGG16 [156], VGG19, and ResNet50 [157] are very popular pre-trained CNN models due to their more indepth architectures [65]. The main reasons are as follows: First, due to the inherent complexity and diversity of breast tissue pathological images, it is not easy to label the images, and the cost of labeling data by medical experts is very expensive. Therefore, there are few publicly available largescale labeled image datasets. However, transfer learning can overcome the problem of small datasets effectively [158]. Secondly, in the classification task of breast histopathology images, most of the pre-trained models are from the Ima-geNet Large Scale Visual Recognition Challenge [159]. They achieve stable performance on specific tasks and can be safely used for transfer learning in breast cancer classification tasks. Finally, the transfer learning process are helpful to improve accuracy or reduce training time [160], which is an important reason for its popularity.

C. ANALYSIS OF THE OUTSTANDING METHODS IN EACH REVIEWED TASK
In different review tasks, there are some excellent methods proposed. For example, in the BreakHis dataset task, the best results are obtained in [81], where a small SE-ResNet model based on the combination of residual module and Squeeze-and-Excitation block is designed, which can effectively reduce model training parameters. Furthermore, a new learning rate scheduler named Gaussian error scheduler is proposed, which can get excellent performance without complicatedly fine-tuning the learning rate. In the Camelyon 2016 dataset task, the best results are obtained in [93]. In order to detect metastatic breast cancer from WSIs, a fast and dense scanning framework is proposed, referred to as ScanNet. ScanNet is implemented based on the VGG-16 network by changing the last three fully connected layers to fully convolutional layers. In the end, faster performance on tumor localization tasks is achieved and even surpasses human performance on WSI classification tasks. In the ICPR 2012 dataset task, the best results are obtained in [120], where a novel deep cascaded convolutional neural network (CasCNN) is designed to detect mitosis. The advantage of the CasCNN is that it can significantly reduce the detection time and achieve satisfactory accuracy. In the Bioimaging 2015 Breast Histology Classification Challenge dataset task, the best results are obtained in [128], where a method for breast cancer classification with the Inception Recurrent Residual Convolutional Neural Network (IRRCNN) model is proposed. The IRRCNN [161], [162] is one of the improved hybrid DCNN architectures based on inception [163], residual networks [157], and the RCNN architectures [164]. Compared with them, the main advantage of this model is that better recognition performance can be achieved using the same number or fewer network parameters.

D. THE POTENTIAL OF THE METHODS MENTIONED IN THIS REVIEW IN OTHER FIELDS
In addition, this review discussed the deep ANNs method not only can be applied in the field of breast histopathological image analysis, but also in the field of other closed microscopic image analysis, such as: Cervical histopathological analysis [165], [166], [167], cervical cytopathological analysis [168], [169], [170], stem cell analysis [171], [172], microbiological image analysis [173], [174], [175], sperm quality analysis [176], [177], [178], and rock microstructural analysis [179], [180]. No matter from the aspects of image pre-processing, feature extraction and selection, segmentation, and classification, or from the aspects of deep ANN model design and proposed framework idea, the methods of deep ANN summarized in this review can bring a new perspective to the research in other fields.

V. CONCLUSION AND FUTURE WORK
In this review, the methods of breast cancer histopathological image analysis based on the artificial neural network are comprehensively summarized, which are grouped into the classical artificial neural network and deep neural network methods. In addition, when summarizing the deep neural network method, the related work is grouped according to the applied datasets. In each dataset, the related works are arranged in ascending chronological order. From classical review works in Sec. II and subsequent analysis in Sec. II-A, it is found that the ANNs used in the BHIA field around 2012 are classical neural networks. In the analysis of histopathological images of breast cancer, MLP and PNN are the most applied classical ANNs. However, they are only used as classifiers. In feature extraction, most of the research is used for texture features and morphological features. Among deep learning based methods whose related works and corresponding analysis are discussed in Sec. III-A and Sec. III-B, deep learning technology, especially deep convolutional neural networks, has made excellent achievements in the classification and segmentation of breast histopathological images, which will help patients with early detection, diagnosis, and treatment of breast cancer. According to the survey, transfer learning methods based on CNN are the most frequently used. But from the review works in Sec. IV-C, improved and novel network frameworks tend to perform better in different datasets.
In the future, there is still room for improvement. First, researchers can combine the characteristics of pathological images to develop a new network model to analyze the histopathological images of breast cancer. Secondly, there is still a lack of large-scale, comprehensive, and fully-labeled WSI datasets. Therefore, the establishment of large public datasets is of great value for future research. Thirdly, the classification system of breast cancer is complex, and there are many subtypes under each lesion type. Studying patterns correlated to molecular subtype, treatment response, and prognosis to refine the diagnosis in precision medicine remains a significant challenge [12]. Finally, GANs are currently used to generate datasets. However, the advantages of microscopic image analysis have not been explored, which will be a research direction with great potential and value in the future.