HAMIL: High-Resolution Activation Maps and Interleaved Learning for Weakly Supervised Segmentation of Histopathological Images

Semantic segmentation of histopathological images is important for automatic cancer diagnosis, and it is challenged by time-consuming and labor-intensive annotation process that obtains pixel-level labels for training. To reduce annotation costs, Weakly Supervised Semantic Segmentation (WSSS) aims to segment objects by only using image or patch-level classification labels. Current WSSS methods are mostly based on Class Activation Map (CAM) that usually locates the most discriminative object part with limited segmentation accuracy. In this work, we propose a novel two-stage weakly supervised segmentation framework based on High-resolution Activation Maps and Interleaved Learning (HAMIL). First, we propose a simple yet effective Classification Network with High-resolution Activation Maps (HAM-Net) that exploits a lightweight classification head combined with Multiple Layer Fusion (MLF) of activation maps and Monte Carlo Augmentation (MCA) to obtain precise foreground regions. Second, we use dense pseudo labels generated by HAM-Net to train a better segmentation model, where three networks with the same structure are trained with interleaved learning: The agreement between two networks is used to highlight reliable pseudo labels for training the third network, and at the same time, the two networks serve as teachers for guiding the third network via knowledge distillation. Extensive experiments on two public histopathological image datasets of lung cancer demonstrated that our proposed HAMIL outperformed state-of-the-art weakly supervised and noisy label learning methods, respectively. The code is available at https://github.com/HiLab-git/HAMIL.

I. Introduction

Tumor micro environment plays an important role in promoting tumor growth [1], [2], and has an influence on prognosis of cancer patients and therapeutic effect [3], [4]. Recognizing different types of tissues in tumor micro environment including tumor epithelial tissue, tumor stromal tissue and normal tissue is critical for accurate diagnosis and treatment decisions of cancers, as they are clinically relevant with tumor progression [2]. Current standard for cancer detection and grading is based on histopathological images, i.e., Whole Slide Imaging (WSI) commonly stained with Hematoxylin and Eosin (H&E). WSI provides high-resolution imaging of the tumor micro environment with giga pixels, and segmentation of the WSI into different tissue types including tumor epithelial and stromal tissues provides quantitative measurements of the tumor region’s size and shape, which is important for accurate diagnosis. In addition, the segmentation results provide Region of Interests (ROI) for down-stream feature extraction that is required in gene expression pattern and prognosis prediction [5], [6]. However, due to the large image size with ambiguous and complex tissue boundaries, manual segmentation is time-consuming and limited by the operator’s experience. Therefore, it is desirable to segment the histopathological images automatically.