Semantic GAN: Application for Cross-Domain Image Style Transfer

Image style transfer has attracted much attention from many fields and received promising performance. However, style transfer in the cross-domain field, e.g., the transfer between near-infrared and visible light images, is rarely studied. In the cross-domain image style transfer, one key issue is mismatching problem existing in the generated semantic regions. In this paper, we propose a novel model of Semantic GAN, which integrates the semantic guidance and the recent CycleGAN. In particular, we present a semantic style loss with Gram matrix to well preserve the semantic information in the generated images. The proposed Semantic GAN can control the transfer in the right way with semantic masks and solve the mismatching problem. We apply our approach to two outdoor scene datasets to evaluate the performance of all competing methods. The experimental results show that our approach outperforms previous methods in addressing the mismatching problem and providing a good quality result.