Text-Image Generation Using Visual-Language Transformer Parallel Decoding

Autoregressive generative models have shown their superiority in the fields of vision and language, such as high accuracy, high fidelity, and strong stability. Most existing methods are trained around text-guided image generation tasks and image-guided text generation tasks. However, not only can these two generative tasks be trained on the same dataset, but both require the alignment of visual and textual representations in the same semantic space. In order to be able to learn better generalization of image and text representations, we unify these two generation tasks in an autoregressive generative model and enhance the quality of text and image generation through bidirectional generation goals. In order to take the 2D-structure of the image into account. To this end, we propose a visual-linguistic Transformer autoregressive generative model. The visual-language Transformer model adopts a novel decoding strategy that takes the 2D-structure of the image into account without affecting the generation of the 1D sequence structure of the text. Through improved training strategies, we can not only generate highly accurate text descriptions, but also enhance the model's image understanding and generate high-quality images.