Apart GAN: Fine Image Restoration for Multilevel Resolution

In order to fully preserve the appearance information of the target human body, such as texture, style, color and face and the pose of the 3D human body model, this paper proposes Apart GAN as a virtual video generation network, firstly, using semantics the segmentation network extracts the mask mapping matrix, and uses the mask mapping matrix to segment the human foreground and the site background of the target video, and proposes a foreground and background feature segmentation loss function for the generated foreground and background. Secondly, the appearance information of the target person is extracted in the feature space, the target image and the generated image feature information of several frames before and after are retained, and a feature mapping function is constructed between the target body and the 3D human body model to restore the external characteristics of the target body, and the body and the heads are restored using different network generation methods. In addition, an optical flow estimation function is constructed between the generated images of the previous frames and the current 3D human body model to achieve the spatial timing of the generated video. For the restoration of the site background, the image background restoration network is used and the dilation matrix is used on the mask mapping matrix to restore a smooth and soft background image. Finally, the restored background and the generated human foreground are combined to obtain the final virtual video.