PESSN: Precision Enhancement Method for Semantic Segmentation Network

Semantic segmentation to understand an image at pixel level is an important problem in the computer vision. In the traditional object detection, each object in an image is detected at its minimum bounding rectangle level, but, in the semantic segmentation, it is detected at pixel level and thus the segmentation result is more flexible and meaningful. However, the characteristics of the semantic segmentation network might make over-segmentation with a few pixels misunderstood and result in the low precision rate. In this paper, we propose a method to enhance the precision rate of the semantic segmentation network. In order to address the over-segmentation, we define confidence-based and semantic-correlation-based outliers. Confidence-based outlier is defined by the confidence value, weighted by the number of segment's pixels, of the semantic segmentation network and semantic-correlation-based outlier is defined by the distance in the Word2Vec space. If a pixel is determined as not only confidence-based but also semantic-correlation-based outlier, the pixel is pruned from the segmentation result. We evaluate the proposed method with the images of COCO dataset and show the f-score, as well as the precision rate, of the semantic segmentation is significantly improved.