A New Deep Segmentation Quality Assessment Network for Refining Bounding Box Based Segmentation

Capturing quality cues in blind segmentation quality assessment (SQA) is a challenging task. This paper proposes a new blind SQA method that captures a variety of quality cues using two considerations: one is to consider different types of data, such as segmentation mask and original image patch and the other is to consider different types of data operations, such as the max pooling operation, the average pooling operation, and the convolution operation. An end-to-end segmentation quality assessment network is proposed to capture these different types of quality cues by the above-mentioned two considerations. Then, since the segmentation result by the traditional bounding box-based segmentation method using fixed parameter setting is usually inaccurate, we use the proposed blind SQA network to refine the traditional bounding box-based segmentation results, with the idea of selecting parameters that are adaptive to each image. A two-step refinement method is proposed. The first step generates multiple segmentation results by various parameter settings. The second step, then, uses the segmentation quality assessment to select the best quality segmentation result as the new result. To train and verify the proposed method, a new dataset constructed by using the PASCAL VOC dataset and the bounding box-based segmentation methods, such as Grabcut and improved FCN method, is constructed. The proposed method is finally verified on the constructed dataset and two tasks: the segmentation quality assessment and the bounding box-based segmentation. The experimental results demonstrate the effectiveness of the proposed method.