InteractionGAN: Image-Level Interaction using Generative Adversarial Networks

We present a novel strategy for image-level interaction that is applicable to a single image without any prior structural knowledge, such as object status or reconstructed 3D models. By training sets of input image and interaction pairs using a target image, our model can generate result images by applying the desired interaction to new unseen images. The proposed method is differentiated from previous approaches for changing poses, which requires absolute statuses for training images or estimated 3D model with reconstruction errors. Based on the conceptual analysis of encoder-decoder networks, we propose a novel generator network architecture containing a feature converter network, which is suitable for applying interactions to images. We also implement a discriminator network for training, which is a well-known technique for generative adversarial networks. Experimental results demonstrate that the proposed method successfully generates result images with applied interactions without any prior knowledge. We expect that our method will provide insights into novel interaction schemes for augmented reality by reflecting interactions onto real scenes and providing more realistic user experiences.