OmniStyle: Attention-Optimized Global and Local Image Stylization with Diffusion Model Inversion

Recent interest in large-scale text-driven diffusion models has highlighted their ability to generate diverse images from textual prompts, with style transfer being a significant application. However, existing text-guided stylization methods are constrained by the reliance on manually created masks for localized transformations, which limits scalability and automation. In this work, a novel framework, OmniStyle, is introduced for text-driven global and localized image style transfer, leveraging diffusion model inversion and attention-based mask generation. Input images are mapped into latent noise representations using DDIM inversion, while cross-attention maps are utilized to automatically generate precise semantic masks, eliminating the need for manual annotations. Latent features are dynamically optimized with mask-guided constraints and attention manipulations, enabling fine-grained style transfer confined to target regions while maintaining the integrity of global content. Extensive experiments demonstrate the scalability and effectiveness of OmniStyle.