Dual-Level Masked Semantic Inference for Semi-Supervised Semantic Segmentation

Semi-supervised semantic segmentation pursues a holistic pixel-wise understanding of unseen images with limited annotation. To this end, existing methods focus on regularizing per-pixel prediction consistency within unlabeled data, while rarely modeling contextual relationships. But in fact, contextual semantics can provide valuable clues for scene understanding like inner-object continuity and spatial relationships' causality. Thus, in this paper, we propose a Dual-level Masked Semantics Inference (DMSI) that takes the initiative to explicitly learn contextual relationships via enforcing our model to infer the semantics of a pixel according to its surrounding contexts. This allows our model to exhaust accurate semantics by incorporating inter-pixel context clues, further leading to comprehensive segmentation. Specifically, DMSI comprises two main components. 1) Dual-level mask consistency regularization (DMCR) that learns the ability of semantics inference by aligning the predictions of masked views with the prediction of the complete view. The masked views here come from both the image level and feature level, where our model captures low-level attributes and high-level representations respectively. 2) AdaMask that provides a proper mask position and ratio for each image, guiding our model to focus on semantic-rich regions while providing balanced training between hard and easy samples. Through learning the ability of semantic inferring, DMSI remarkably enhances the interaction between pixels, further progressively intensifying the understanding of semantics. Extensive experiments under various settings on Cityscapes and Pascal VOC 2012 show that DMSI achieves new state-of-the-art performances. Furthermore, analysis indicates that our method has superiority in mining inter-pixel semantic relationships and improving robustness facing noise corruption.