Attention Mechanism and Out-of-Distribution Data on Cross Language Image Matching for Weakly Supervised Semantic Segmentation

The fully supervised semantic segmentation requires detailed annotation of each pixel, which is time-consuming and laborious at the pixel-by-pixel level. To solve this problem, the direction of this paper is to perform the semantic segmentation task by using image-level categorical annotation. Existing methods using image level annotation usually use CAMs (Class Activation Maps) to find the location of the target object as the first step. By training a classifier, the presence of objects in the image can be searched effectively. However, CAMs appear that (1) objects are excessively focused on specific regions, capturing only the most prominent and critical areas and (2) it is easy to misinterpret the frequently occurring background regions, the foreground and background are confused.

This paper introduces CLODA, the concept of double branching in the cross language image matching framework, and adds a convolutional attention module to the attention branch to solve the problem of excess focus on objects in the CAMs. Importing out-of-distribution data on out of distribution branches helps classification networks improve misinterpretation of areas of focus. Optimizing regions of interest for attentional branch learning using cross pseudo supervision on two branches. Experimental results show that the pseudo masks generated by the proposed network can achieve 75.3% in mIoU with the PASCAL VOC 2012 training set. The performance of the segmentation network trained with the pseudo masks is up to 72.3% and 72.1% in mIoU on the validation and testing set of PASCAL VOC 2012.