Fine-Tuning Text-to-Image Generation Models Using Curriculum Learning for Yao Costume Image Generation

With the emergence of diffusion models, synthesizing images from textual descriptions has become an active research field. However, current descriptions often lack specificity, focusing mostly on natural images rather than attire of Chinese ethnic minorities like the Yao group. Additionally, text-to-image tasks typically demand a large number of annotated pairs, making dataset creation costly. Leveraging multimodal pre-training, we propose a curriculum learning approach to fine-tune stable diffusion models, enabling high-quality Yao costume image generation with color, texture, and pattern fidelity. Through a curriculum-based loss function, we ensure semantic consistency between generated images and text descriptions. Ablation experiments and comparisons with DreamBooth demonstrate the superior performance of our method in terms of both image quality and semantic consistency.