Diversified Image Inpainting With Transformers and Denoising Iterative Refinement

Image inpainting is a long-standing key problem in the field of computer vision, which aims to fill the missing parts of an image with visually realistic and semantically appropriate content. For a long time, in the research work at home and abroad, how to generate diverse and realistic images is a dilemma faced by image inpainting. With the continuous iteration of deep learning technology, transformer feature extraction model and a new generation paradigm, diffusion model, are emerging in vision tasks. Attention-based transformer models can effectively model long-distance dependencies and flexibly design output content. The diffusion model is stable in training, and the quality of its generated images is already better than that of generative adversarial networks. We decompose the inpainting problem into two key steps: diversified pre-generation and high-resolution reconstruction. Firstly, referring to the discretized pixel set, a transformer information association model is designed to sample from the granularity of pixel values, so as to obtain low-resolution results with diverse appearance. Then, a denoising diffusion model is used to reconstruct the high-resolution image, which is a conditional and iterative refinement process. Ultimately, we achieve a set of image restoration methods that produce diverse results and support high-fidelity output.