Adaptive multimedia playout method based on semantic structure of media stream

Real-time multimedia communication over IP has been put into practical use in recent years. Real-time multimedia communication requires clock frequency synchronization between the transmitter and the receiver for the receiver to playout the received media correctly. However, it is generally difficult to achieve clock synchronization between the transmitter and the receiver over the IP network environment due to large delay jitter. Also, there are many information appliances without clock adjustment for synchronization, including software-based decoders on personal computers. We propose a novel real-time adaptive media playout method; it is highly effective and allows appropriate playout and presentation of real-time media. Its most significant feature is that clock synchronization is NOT required. Our adaptive playout method takes advantage of the semantic structure of media streams. We regard each media stream as consisting of components such as "semantic block", "vacant block" and "adaptation unit". The receiver playouts the media in such a way that each semantic block is appropriately preserved to protect continuity of content semantics, with additional playout adjustment, such as adjusting playout timing at semantic block boundaries, and reducing or extending vacant blocks and/or adaptation units. This playout with adjustments causes almost no unfavorable effects on the quality of media presentation because the semantic structure of each media stream is appropriately preserved.