Spatio-temporal model-assisted very low-bit-rate coding with compatibility

We propose the concept of spatio-temporal model-assisted compatible coding, a technique that selectively encodes areas of different importance to the human eye in terms of space and time in moving images. Our motivation is the fact that eye contact and lip synchronization are very important in person-to-person communication. Several areas including the eyes and lips require different types of quality since they have different perceptual significance to human observers. For example, for the eyes, high resolution is needed for clear communication, while for the lips frequent refresh is desirable. Fully automatic algorithms are described for performing model area detection and motion tracking. An area-selective (AS) rate control scheme is proposed, in order to effect AS bit allocation. It extends prior work that dealt with only the spatial dimensions to include the temporal aspects of the video signal as well. The decoder does not need to be changed in any way although the encoder's rate control unit is modified. As a result, the scheme can be used in compliance with standards-based codecs. As the scene model is only used in the rate control module (hence the term "model-assisted"), the technique does not suffer from the artifacts associated with model-based coding. Extensive experimental results are provided using ITU-T H.263, addressing very low bit-rate compression (below 64 kb/s).