Smart video streams: 101 uses of the user data field in MPEG

MPEG specifies private user data fields (UDF) in the standard syntax. We unfold the potential capabilities of UDF usage. We have developed a testbench to experiment with manipulation functions for this user data field. We introduce the notion of "smart" video streams which carry additional information in the UDFs. We propose a variety of applications which can benefit from efficiently using the private data space in MPEG. Three example application areas are discussed: database applications, networked video applications and increasing the error resiliency of transmitted video. We provide an analysis of the efficiency of smart streams. Our experimental results show that by introducing a small overhead the video streams can carry important additional information.