Class-Specific Affinity based Weakly Supervised Semantic Segmentation with Neutral Region Exploration

Image-level weakly supervised semantic segmentation (WSSS) reduces the cost of semantic segmentation significantly as only category labels are required. In the WSSS pipeline, the initially obtained seed areas are imprecise and should be refined. AffinityNet is a widely used seed area refinement method which learns pixel-level affinity between coordinates to adjust the object regions. However, the capabilities of AffinityNet are limited due to the complex simultaneous exploration of the affinities of all object classes, as the classes have different characteristics. This paper proposes a class-specific AffinityNet (CSANet) in which the affinities of each class are learnt separately by the class-specific affinity layers. Moreover, as the learning task is simple because the affinity of only one class is focused, the information contained by the uncertain neutral regions is extracted to provide additional information in the training of the class-specific affinity layers. The empirical results demonstrate the effectiveness of our method, which captures more precise boundaries and significantly improves segmentation results on the PASCAL VOC 2012 segmentation benchmark.