A Language-Free Evolutionary Framework for Text-to-Image Generation

Computational creativity is an emerging topic in artificial intelligence area. Past studies have shown the effectiveness of deep learning methods for image generation. Contrast to simple image generation, it is more practical to generate images according to specific description, known as text-to-image generation. However, text-to-image generation is much more difficult than simple image generation as it may involve natural language processing for semantic retrieval, learning mapping between images to semantics, and learning image generator. This study proposes an evolutionary framework for language-free text-to-image generation. There are three main features in the proposed framework. First, the framework bases on language-free model for image generation, which purifies the text-to-image generation problem as an optimization problem over a learnt image generator. Second, the use of language-free model simplifies the requirements of data preparation and learning process. Third, in the proposed framework an objective function is designed for improving the quality of the generated images in terms of both photo realisticness and semantic consistency. Experimental results validate the effectiveness of the proposed framework in text-to-image generation on different scenarios, in comparison to state-of-the-art methods. This study further provides and examines the possibility for the proposed framework on image manipulation as a potential application. This study also discusses the designed operators and hyper-parameters and provides guidelines for the settings.