Instance Map Based Image Synthesis With a Denoising Generative Adversarial Network

Semantic layout-based image synthesizing, which has benefited from the success of generative adversarial networks (GANs), has received a substantial amount of attention recently. How to enhance the synthesis image equality while maintaining the stochasticity of the GAN remains a challenge. We propose a novel denoising framework to handle this problem. The generation of overlapping objects is another challenging task when synthesizing images from a semantic layout to a realistic RGB photograph. To overcome this deficiency, we include a one-hot semantic label map to force the generator to pay more attention to the generation of overlapping objects. Furthermore, we improve the loss function of the discriminator by considering the perturbed loss and cascade layer loss to guide the generation process. We applied our methods to the Cityscapes, photo-sketch, day-night, facades, and NYU datasets to demonstrate the image generation ability of our model.