Rate-distortion optimization for scalable multi-view video coding

In recent years, multi-view/3D video applications, such as three-dimensional television (3DTV) and free-viewpoint television (FTV), have drawn increasing attention. Since the amount of data that has to be stored or transmitted increases proportionally with the number of cameras, efficient compression of multi-view/3D video is crucial. Scalable multi-view video coding is one of the methods to address this challenge. But, in streaming multi-view/3D video over a network to heterogeneous receivers, efficient video compression while maintaining a high quality of received video is very challenging. This paper presents a novel method for rate-distortion optimization in scalable multiview video. We apply the Karush-Kuhn-Tucker (KKT) conditions in minimizing the perceptual distortion of decoded video under the conditions that the sum of bits generated from different views is constrained within a given bit budget. Since the constraint-based optimization problem is usually computational intensive, our proposed approach considers the concept of disparity between layers and disparity between views to reduce this computational complexity. Simulation results indicate that the proposed approach is able to meet network bandwidth limitations with acceptable overall video quality.

1. Introduction

A typical multi-view/3D sequence consists of multiple videos captured simultaneously from the same scene by several cameras that are located at different positions. Multiview video coding is one of the latest amendments to the H.264/AVC standard [1] that allows us to compress multi-view/3D video streams more efficiently. But even with the improvement of MVC, multi-view/3D video is currently beyond the capabilities of most mobile devices and environments. One approach to reduce the amount of data is the scalability of the coded bitstream, which is particularly useful for the transmission of multi-view/3D video in heterogeneous environments where receivers have different bandwidth, display size, and processing power. Scalable Video Coding (SVC), in single view video, enables the decoding of partial bitstreams (called layers) to provide video sequences with lower temporal/spatial resolutions or reduced quality. Typically, a “Base Layer” (BL) is built which carries the minimum amount of video information that is necessary. One or more “Enhancement Layers” (EL) can then be built to increase the quality on top of the BL.