Ablation Studies in Activation Maps for Explainable Semantic Segmentation in Industry 4.0

In recent years, much attention has been given to various eXplainable Artificial Intelligence (XAI) and interpretability methods. Their extension to dense prediction tasks, however, has been underexplored. Gradient-based saliency maps, highlighting feature importance in terms of input pixels, have been frequently used as fast and simple visual explanation techniques. Nonetheless, they face several problems, and the exploration of different types of attribution methods is warranted. In this paper, we investigate gradient-free semantic segmentation explanations that are based on ablating activation maps. We explore their potential for industrial applications, specifically for fruit pitting machines. We also extend the application of Ablation-CAM, a gradient-free ablation-based interpretability technique, to semantic segmentation. Finally, we discuss the sensitivity of activation maps to partial occlusions of either the foreground or the background class regions.