NSGA-II-DL: Metaheuristic Optimal Feature Selection With Deep Learning Framework for HER2 Classification in Breast Cancer

Immunohistochemistry (IHC) slides are graded for breast cancer based on visual markers and morphological characteristics of stained membrane regions. The usage of whole slide images (WSIs) from histology in digital pathology algorithms for computer-assisted evaluations has increased recently. Human epidermal growth factor receptor 2 (HER2)-stained microscopic images are challenging, time-consuming, and error-prone to evaluate manually. This is due to different staining, overlapped regions, and huge, non-homogeneous slides. Additionally, the classification of HER2 images by the selection of fundamental features must be used to capture the difficult elements of the images, such as the irregular cell structure and the coloring of the tissue of the cells. To solve the above problems, a transfer learning model-based, trainable metaheuristic method for choosing the best features is suggested in this paper. Moreover, the suggested model is efficient in reducing model complexity and computational costs as well as avoiding overfitting. The four main components of the proposed cascaded design are: (1) converting WSIs to tiled images and enhancing contrast with fast local Laplacian filtering (FlLpF); (2) extracting features with a ResNet50 CNN technique based on transfer learning; (3) selecting the most informative features with the help of a non-dominated sorting genetic algorithm (NSGA-II) optimizer; and (4) using a support vector machine (SVM) to classify HER2 scores. Results from the HER2SC and HER2GAN datasets show that the suggested model is superior to other methods already in use, with 94.4% accuracy, 93.71% precision, 98.07% specificity, 93.83% sensitivity, and a 93.71% F1-score for the HER2SC dataset being achieved.