Enhanced Macroblock Features for Dynamic Background Modeling in H.264/AVC Video Encoded at Low Bitrate

While H.264 is a well-established standard for video surveillance, its high-profile implementation, in particular, has the unique capabilities that pack more visual detail into a given bitrate. Several algorithms exist that detect moving objects from the H.264 bitstream, most of which end up incorrectly classifying the nonstationary or dynamic components (e.g., jitter camera, waving trees, ripples, etc.) of the background as foreground. Moreover, the coarse quantization schemes adopted at constrained bitrates pose new challenges for direct identification of moving objects/targets from the bitstream. This paper focuses on dynamic background modeling for the H.264 video encoded at very low bitrates. To this end, enhanced features of the fundamental coding unit, i.e., the macroblock (MB) are proposed. Based on the temporal statistics of the MB features gathered from initial frames, MBs that potentially contain parts of moving objects are selected in subsequent frames. The selected MBs constitute a coarse segmentation map of the foreground at the MB level. Finally, pixel-level segmentation of the foreground is obtained by comparing pixels constituting the selected MBs with the colocated counterparts of a background frame. Experimental results showing the comparison of bitstreams encoded at strikingly low bitrates are obtained over a diverse set of standardized surveillance sequences.