Affine Transformation-Based Generative Face Video Compression

In this paper, we propose a generative face video compression framework based on affine transformations to better represent large movements without parameter transmission. It mainly consists of an encoder and decoder, and our encoder is similar to the one in [1]. Intra frame are compressed by the existing encoder, while subsequent inter frames are compressed into compact inter frame features. In the decoder, feature alignment is first established to map the decoded intra frame and inter frame features into the same domain. The aligned features are then combined with the appearance features extracted by the appearance encoder from the intra frame and fed into the coarse-fine affine transform module to establish motion estimation and compensation. The coarse affine transform focuses on global motion, while the fine affine transform deals with local motion, such as lip motion. Finally, the transformed features are fed into the image generation module to obtain the final reconstruction results.