Abstract:
Traditional approaches for semantic image synthesis mainly focus on text descriptions while ignoring the related structures and attributes in the original images. Therefo...Show MoreMetadata
Abstract:
Traditional approaches for semantic image synthesis mainly focus on text descriptions while ignoring the related structures and attributes in the original images. Therefore, some critical information, e.g., the style, backgrounds, objects shapes and pose, is missed in the generated images. In this paper, we propose a novel framework called Conditional Cycle-Generative Adversarial Network (CCGAN) to address this issue. Our model can generate photo-realistic images conditioned on the given text descriptions, while maintaining the attributes of the original images. The framework mainly consists of two coupled conditional adversarial networks, which are able to learn a desirable image mapping that can keep the structures and attributes in the images. We introduce a conditional cycle consistency loss to prevent the contradiction between two generators. This loss allows the generated images to retain most of the features of the original image, so as to improve the stability of network training. Moreover, benefiting from the mechanism of circular training, the proposed networks can learn the semantic information of the text much accurately. Experiments on Caltech-UCSD Bird dataset and Oxford-102 flower dataset demonstrate that the proposed method significantly outperforms the existing methods in terms of image details reconstruction and semantic information expression.
Date of Conference: 20-24 August 2018
Date Added to IEEE Xplore: 29 November 2018
ISBN Information:
Print on Demand(PoD) ISSN: 1051-4651
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Image Synthesis ,
- Semantic Image Synthesis ,
- Image Features ,
- Semantic Information ,
- Textual Descriptions ,
- Stable Training ,
- Cycle Consistency ,
- Cycle Consistency Loss ,
- Interpolation ,
- Convolutional Neural Network ,
- Input Image ,
- Generative Adversarial Networks ,
- Baseline Methods ,
- Source Images ,
- Residual Block ,
- Image Domain ,
- Synthetic Images ,
- Batch Normalization Layer ,
- Variational Autoencoder ,
- Realistic Images ,
- Conditional Generative Adversarial Network ,
- Sentence Embedding ,
- Latent Vector ,
- Fake Images ,
- Image X ,
- Style Image ,
- Series Of Layers ,
- Active Layer ,
- Convolutional Layers
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Image Synthesis ,
- Semantic Image Synthesis ,
- Image Features ,
- Semantic Information ,
- Textual Descriptions ,
- Stable Training ,
- Cycle Consistency ,
- Cycle Consistency Loss ,
- Interpolation ,
- Convolutional Neural Network ,
- Input Image ,
- Generative Adversarial Networks ,
- Baseline Methods ,
- Source Images ,
- Residual Block ,
- Image Domain ,
- Synthetic Images ,
- Batch Normalization Layer ,
- Variational Autoencoder ,
- Realistic Images ,
- Conditional Generative Adversarial Network ,
- Sentence Embedding ,
- Latent Vector ,
- Fake Images ,
- Image X ,
- Style Image ,
- Series Of Layers ,
- Active Layer ,
- Convolutional Layers