Sketch-based Deep Generative Models Conditioned on a Background Image

Thanks to modern image generation techniques using deep learning, we can easily create images from simple sketches. Unfortunately, previous work focuses on generating a foreground only and does not consider composing the foreground and an arbitrary background image. Therefore, it suffers from unnaturalness arising from a composition between the generated foreground and the original background image. In this paper, we present a method that not only generates a foreground image from sketches but also composes the foreground and background naturally using a generative adversarial network (GAN). In particular, we propose three approaches to conditioning the GAN on sketches and a background. Evaluation experiments confirm that the proposed models obtain different results according to our design concepts and can synthesize more natural images than a baseline method.