Ultra-low latency video codec for video conferencing

Video codec (e.g. HEVC, H.264, H.263, H.261) are used for real time video conferencing over internet. The amount of latency from end to end (or round trip) has significant impact on perceived quality of video call. This paper explains overall latency for entire signal chain with focus especially on video codec. The paper explains typical configuration to optimize overall latency of video processing down to range of 1 video frame processing time. The paper proposes new sub-frame based data flow to cut down overall latency to significantly to fraction of video frame. The paper proposes new design for video codec engine to enable sub-frame based data flow consisting of novel way of exchanging data between entropy engine and application, pre-fetching of video data without stalling video performance and sending of partial video output to network. The overall design enables reduction of processing latency of video engine from multiple frames to few lines of video. The overall solution on TI's Davinci series (DM816x) device achieves latency up-to 2 msec compared to prior art measurement of 33 msec resulting in better user experience due to large improvements in perceived visual quality.