Loading [a11y]/accessibility-menu.js
SketchScene: Scene Sketch To Image Generation With Diffusion Models | IEEE Conference Publication | IEEE Xplore

SketchScene: Scene Sketch To Image Generation With Diffusion Models


Abstract:

Sketch is an abstract visual representation that can be recovered as natural photographs in the human mind. Many researchers are drawn to work on translating abstract ske...Show More

Abstract:

Sketch is an abstract visual representation that can be recovered as natural photographs in the human mind. Many researchers are drawn to work on translating abstract sketches to natural photographs. Since conventional sketch-to-image models are designed to generate images with a single object as the subject, generating scene image with multiple classes of objects is a tricky problem. To tackle this challenge, we propose the first scene sketch-to-image generation method based on diffusion models. Our model uses an encoder to summarize the contour and class features of the scene sketch into a latent variable, and a decoder to reconstruct scene images from it. In scene sketch-to-image generation tasks, our method outperforms the state-of-the-art methods. Experiments also show that our model beats other methods in zero-shot general sketch-to-image generation. It demonstrates our model’s potential for full-domain image generation.
Date of Conference: 10-14 July 2023
Date Added to IEEE Xplore: 25 August 2023
ISBN Information:

ISSN Information:

Conference Location: Brisbane, Australia

References

References is not available for this document.