Artsy-GAN: A style transfer system with improved quality, diversity and performance

This paper proposes Artsy-GAN: a generative adversarial approach for style transfer. Style transfer has focused mostly on transferring the style of one image (e.g. painting) to another image (e.g, a photograph). Important progress has been done to process any image in real-time and, more recently, with arbitrary style images. A different approach has been proposed based on Generative Adversarial Networks (GAN), by translating an image from one context (e.g. photograph) to another (e.g. Van Gogh painting). To achieve this image-to-image translation, for example, Cycle-GAN uses a cycle consistency requirement to be able to recover the original image after translation and thus keep the content from the input images. This is complex and slow to train. Another disadvantage of this systems is that they take the source of randomness only from the input image, limiting the diversity of the output. In this work, we improve the quality, efficiency and diversity in three ways. First, we use perceptual loss to replace the reconstructor with significant improvement in quality and speed of training. Second, we improve the speed for predicting by processing images with chroma sub-sampling. Third, we improve diversity by introducing noise in the input of the generator and a new loss function that encourages to generate different details for the same content image. Experiment results show that, compared to the state-of-art, Our method could improve the quality and diversity of the output, as well as the speed advantage.