Image Semantic Segmentation Based on the GAN Auxiliary Network

To enhance the performance of existing deep neural networks in semantic segmentation while preserving efficiency at the same time, a semantic segmentation network with the help of GAN (Generative Adversarial Networks) is proposed. The method consists of a generator and a discriminator. The segmentation results obtained from the generator are encoded and then fed into the discriminator to obtain the pixel-wise uncertainty values. Such uncertainty values are taken as weights for the calculation of CEGU (Cross-Entropy with GAN Uncertainty) to help the optimization of the generator. The discriminator is removed after training. Experiment results show that the mean IoU (Intersection over Union) scores of the segmentation results grow by 4.7% and 3.2% respectively on ResNet-50 and ResNet18, after using the GAN auxiliary method along with the CEGU. It shows that such a GAN auxiliary network can significantly improve the performance of basic end-to-end methods with various backbones on the semantic segmentation task, without introducing extra computation cost in the test phase.