ST-Double-Net: A Two-Stage Breast Tumor Classification Model Based on Swin Transformer and Weakly Supervised Target Localization

Breast cancer is the second deadliest cancer (after lung cancer) globally among women, with high incidence and mortality rates. Its early diagnosis is pivotal for improving the cure rate. With the continuous development and maturity of deep learning technologies, traditional classification models have been widely applied for automated classification of pathological images. However, several challenges still persist. For instance, traditional classification models typically perform well in processing images with clear distinctions between target objects and backgrounds, but struggle to accurately classify pathological images due to the lack of clear distinctions between tumor lesion areas and background areas. In the light of this, we propose a two-stage breast tumor pathological classification model based on weakly supervised target localization, named ST-Double-Net. In the proposed model, precise lesion localization and classification are achieved in two stages. In the first stage, a set of global feature maps is obtained by utilizing the Swin Transformer. These feature maps are then input into a newly designed heatmap cropping (HMC) module, which forces the model to focus on discriminative features of lesion areas through heatmap-guided cropping, without requiring bounding boxes or relevant annotation information. This gradual refinement of target localization facilitates the extraction of useful global features, from coarse to fine. The images with discriminative features generated in the first stage serve as inputs for the second stage, where another Swin Transformer extracts local features from the magnified lesion region images. Finally, the global and local features extracted in the first and second stage, respectively, are fused to emphasize subtle differences in the images, thereby enhancing the model’s classification ability. The proposed ST-Double-Net model is evaluated on the BreaKHis and BACH public datasets, demonstrating superior performance compared to state-of-the-art models.