Compression of Time-Varying Textured Meshes using Patch Tiling and Image-based Tracking

This paper presents our answer to the dynamic mesh compression Call for Proposals (CfP) that was recently launched by the MPEG 3D Graphics Coding (MPEG-3DGC) group. The proposed method begins with a per-frame mesh decimation based on quadric error metrics. The decimated mesh geometry and topology are then encoded per frame, by using a state-of-the art static mesh coder (such as Draco, for example). The associated texture map coordinates (uv) can either also be directly encoded using the same static mesh coder, or an optional mesh segmentation into sub-meshes ("patches") of approximately equal size is applied. For each patch, a new local uv parameterization is computed, meaning that the original uv texture coordinates do not need to be encoded, as the same parameterization is recomputed by the decoder. The patches are then organised into a regular grid in a global uv coordinate system, and the input mesh texture map is reprojected onto the corresponding patch tiles. Intra-frame reorganisation of the texture tiles can then be applied in the first frame of each Group of Pictures (GOP), followed by inter-frame reorganisation for the other frames in the same GOP based on tile tracking using image-based distance metrics. These reorganisations improve spatial and temporal correlations in the reprojected texture maps, allowing these texture maps to be more efficiently coded by standard 2D video codecs than the original texture maps. The proposed method shows notable improvements in rate-distortion performance over the anchor codec used in the CfP, both for the geometry and colour coding, and for the all-intra and random-access lossy coding test conditions.