Semi-Regular Geometric Kernel Encoding & Reconstruction for Video Compression

Conventional video coding schemes employ a hybrid motion prediction / residual transform coding paradigm, which only exploits redundancy in individual pairs of video frames for compression gain. However, rigid geometric structures in 3D space—e.g., a building in a scene’s background—persist across time in a large frame group. Thus if one can extract and encode the geometric structure, then redundancy across the entire frame group can be removed in one shot. In this paper, we extract a best-fitting "semi-regular" geometric structure from a target spatial region in a frame group, which is encoded separately as a unified signal predictor for these frames. By semi-regular, we mean its geometry is simple enough that its shape parameters can be encoded cheaply. This semi-regular structure kernel approximates the 3D shape of an object in the video, on which we project pixels from the frame group to a carefully spaced 2D grid overlaid on the kernel. We encode the projected pixels as an intra-frame using HEVC. The decoded pixels are then back-projected to each frame as the predictor, and the resulting prediction residuals are transform-coded. Experimental results show that employing a semi-regular geometric kernel—a folded 2D plane in our realization—improves coding performance over native HEVC implementation and our previous regular kernel based scheme.