Modular and Lightweight Networks for Bi-Scale Style Transfer

With the emergence of deep perceptual image features, style transfer has become a popular application that repaints a picture while preserving the geometric patterns and textures from a sample image. Our work is devoted to the combination of perceptual features from multiple style images, taken at different scales, e.g. to mix large-scale structures of a style image with fine-scale textures. Surprisingly, this turns out to be difficult, as most deep neural representations are learned to be robust to scale modifications, so that large structures tend to be tangled with smaller scales. Here a multi-scale convolutional architecture is proposed for bi-scale style transfer. Our solution is based on a modular auto-encoder composed of two lightweight modules that are trained independently to transfer style at specific scales, with control over styles and colors.