A New Progressive Multi-stage Diffusion Model for Enhanced Sketch-to-Fashion Image Translation Method

Diffusion models have demonstrated outstanding performance in image translation tasks, but recent studies reveal that their generalization capability is limited when handling datasets containing images from multiple categories. This results in issues such as image blurriness, errors in texture details, and inconsistencies in shape between sketches and the translated images. To address the challenges in current garment image translation methods, a label-free, sketch-based multi-stage image translation method is proposed. The method first uses a conditional diffusion model to generate grayscale images. Then, through an attention mechanism, the generated grayscale image is progressively fused with the sketch. After that, the LeViT model is used to extract the fused image's features. Finally, the conditional diffusion model is applied again to generate a realistic image using the grayscale image and the fused features. By constructing an effective diffusion network, the model captures the intrinsic relationships between the sketch and the target image, achieving progressive translation from sketch to grayscale to the final image. Extensive experiments were conducted on a dataset containing four different categories and over 30,000 image pairs. The results show that, compared to existing image translation methods, the proposed method significantly improves both the quality and diversity of generated images.