Texture Brush for Fashion Inspiration Transfer: A Generative Adversarial Network With Heatmap-Guided Semantic Disentanglement

Automatically accomplishing intelligent fashion design with certain ‘inspiration’ images can greatly facilitate a designer’s design process, as well as allow users to interactively participate in the process. In this research, we propose a generative adversarial network with heatmap-guided semantic disentanglement (HSD-GAN) to perform an ‘intelligent’ design with ‘inspiration’ transfer. Our model aims to learn how to integrate the feature representations, from the styles of both source fashion items and target fashion items, in an unsupervised manner. Specifically, a semantic disentanglement attention-based encoder is proposed to capture the most discriminative regions of different input fashion items and disentangle the features into two key factors: attribute and texture. A generator is then developed to synthesize mixed-style fashion items by utilizing the two factors. In addition, a heatmap-based patch loss is introduced to evaluate the visual-semantic matching degree between the texture of the generated fashion items and the input texture information. Extensive experimental results show that our proposed HSD-GAN consistently achieves superior performance, compared to other state-of-the-art methods.

I. Introduction

For a fashion designer, sources of inspiration can be anything visual, from the buildings of Ancient Rome to a plate of baked beans. Experienced designers seek intricate details and textures from these visual objects that can then be altered, collated, and brought up-to-date to sit within design collections. Raster or vector software, e.g., Photoshop or Adobe Illustrator, are commonly utilized to assist designers in accomplishing their working process. However, these traditional tools can only produce astonishing and realistic imagery if guided by experienced designers; they are unable to automatically create certain ‘inspiration’ fashion designs from visual objects. Therefore, it is highly desirable to devise an automatic fashion design method to assist fashion designers in completing the ‘inspiration’ design process. Fortunately, recent advances in big data analysis and deep learning techniques have provided powerful tools for making designs with this ‘inspiration’ transfer feasible.