Photographic Image Synthesis with Highway Residual U-net

Photographic image synthesis is a new research focus in the field of deep learning, which uses the known image description information to generate the image approaching to the real scene. In this paper, a novel photographic image synthesis method based on the highway residual U-net (HRU) is proposed. The proposed highway residual blocks (HRBs) are embedded between the levels of the U-net and the proposed HRBs effectively filter the information transmitted by each level of the U-net and speed up the convergence of the network. In addition, the resize-convolution layers in HRU are used to replace the deconvolution layers to reduce the checkerboard artifacts in the synthesized images. HRU can be trained end-to-end with the image description map and the corresponding photographic image. Extensive experiments on Cityscapes dataset and GTA5 dataset demonstrate that images synthesized by the presented approach are considerably more realistic than other synthetic approaches.