SegFormer Model in Mammography Lesion Segmentation: A Study on the Impact of GLAM Saliency Maps

This paper explores the potential improvement in the SegFormer model's performance for lesion segmentation in mammograms (MGs) by incorporating saliency maps from the GLAM model. The GLAM model was trained on a million MGs, thus it is reasonable that the model demonstrates a broader scope of generalization compared to the models trained on just a few hundred or thousand images (which are available in the open datasets). Consequently, the GLAM model outputs can be considered as robust input features. The study was conducted by comparing the performance of the SegFormer model trained on i) exclusively on MGs (referred to as “only MG”), ii) a combination of saliency maps and MGs (referred to as “combined”) and iii) exclusively on saliency maps (referred to as “only saliency”). The findings suggest that despite the GLAM model being pretrained on a significant number of MGs, the saliency maps it generated did not enhance the segmentation task. Instead, they introduced uncertainty for both the saliency-only and combined models. This led to an average F1 score of 25.65% and 49.91%, respectively, in comparison to the only MG model, which achieved a higher score of 52.95%.