• Abstract

# Unequal Error Protection for Backward Compatible 3-D Video Transmission Over WiMAX

In this paper, an Unequal Error Protection (UEP) scheme for the transmission of 3-D (Three-Dimensional) video over WiMAX communication channel is proposed. The colour plus depth map stereoscopic video is coded with backward compatibility using a Scalable Video Coding (SVC) architecture, where users with conventional video decoders/receivers can receive the conventional 2-D (Three-Dimensional) video stream whereas users with SVC decoders/receivers and necessary 3-D video displays may render 3-D video. The proposed error protection scheme is based on the perceptual importance of the coded 3-D video components. The UEP method allocates more protection to the colour video packets than the depth map packets in order to receive good quality 2-D/3-D video. The protection levels are assigned through allocating differentiated transmission power for colour and depth map video packets during transmission. On the fly power allocation is based on the estimated distortion of the colour image sequence. The objective and perceptual quality evaluations show that the proposed UEP scheme improves the quality of 2-D video while achieving pleasing quality for 3-D viewers.

SECTION I

## Introduction

THE data losses that occur during transmission affect the reconstructed quality of 2-D video. The effects of transmission errors on the perceived quality of 3-D video could not be less than for the equivalent 2-D video applications, because the errors will influence several perceptual attributes (e.g., naturalness, presence, depth perception, eye-strain, viewing experience, etc), associated with 3-D viewing. Therefore, 3-D video content needs to be protected when transmitted over unreliable communication channels.

The initial deployment of 3-D video services should ideally be backward compatible with 2-D video applications, as most consumers only have 2-D video receivers [1]. In the case of backward compatible 3-D video applications, conventional 2-D video users may receive poor quality video if the components of 3-D video are equally protected. For example, equally protected colour and depth map video transmission may introduce more errors to the colour video, which is viewed by the larger percentage of consumers. Therefore, the protection levels can be unequally allocated, giving more priority for the important data. This paper proposes a UEP scheme for colour plus depth map 3-D video transmission over WiMAX through allocating different transmission power to individual bit-streams according to their perceptual importance.

UEP is one of the most effective techniques for addressing the quality degradation caused by channel errors. This technique focuses on providing added protection for the most important component of the coded data. The UEP schemes which exploit the characteristics of 2-D video are common in research literature. For example, Hannuksela et al. propose an UEP scheme based on Region Of Interest (ROI) coding [2]. The proposed UEP schemes for scalable video streaming over wireless communication channels [3], [4] take the error sensitivity of each coded layer into consideration to provide unbalanced protection. UEP schemes using the perceptual importance of 3-D video have not been thoroughly investigated to date. A prioritized left-and-right view stereoscopic video streaming with estimated distortion and UEP is discussed in [5]. Moreover, the study described in [6] proposes an UEP scheme for progressively transmitted 3D models based on their importance to the decoded model quality. An UEP scheme using redundant motion information (i.e., more protection for the motion information of the coded data) for colour and depth map stereoscopic video streaming is described in [7]. Most of the UEP schemes are implemented with the use of channel coding. However, this introduces more overhead to the encoded bit-stream. The amount of overhead data (i.e., channel coding) added to the encoded 3-D video bit-stream will be much higher than for conventional 2-D video due to the availability of more than one video bit-stream in coded 3-D video. Moreover, this will place more demand on system resources such as bandwidth. Therefore, alternative concepts for achieving UEP have emerged, such as allocating different transmission powers to different parts of the video. For example, the study described in [8] allocates different transmission powers to individual bits according to their bit error sensitivities. Moreover, this method allows optimum use of allocated power for the video service. The UEP scheme proposed in this paper achieves different levels of protection through allocation of unequal transmission power for the 3-D video components. With this approach, the 3-D video packets with more power are more likely to survive than the packets with less power. The coded 3-D video data is categorized based on perceptual importance and distortion estimates before allocation of unequal error protection levels.

This paper is organized as follows. Section II describes the proposed UEP scheme for 3-D video in detail. The power allocation module associated with this method is also presented. The experimental work and results are discussed in Section III. Section IV concludes the paper.

SECTION II

## The Unequal Error Protection Scheme

The colour and depth map based 3-D video representation is widely utilized in research and standardization activities due to its simplicity and adaptability [1], [9], [10]. The coded colour image sequence is projected to the 3-D space, based on the depth map information, using the Depth Image-Based-Rendering (DIBR) method described in [11]. The colour video is the only texture information that is directly viewed by the users. Therefore, the loss of colour video packets will be more annoying to users (i.e., both 2-D and 3-D viewers) than the loss of depth video packets. Furthermore, the colour video itself provides certain depth clues (e.g., identification of objects in the foreground and background) during 2-D viewing. Therefore, the colour video stream should be allocated more protection compared to the depth map packets in general.

Accurate depth maps are required to generate good quality stereoscopic video using the DIBR technique. However, the quality of the depth map does not need to be very high to deliver the good quality 3-D video. Coarsely quantized depth images achieve similar stereoscopic video quality compared to 3-D video rendered with good quality depth images [12]. Furthermore, the effect of depth map transmission errors on the quality of reconstructed 3-D video is insignificant compared to the quality degradation due to the loss of colour video packets [12]. The study in [12] also highlights the importance of prioritizing colour video data ahead of depth map information in order to improve the perceived 3-D video quality. Moreover, the quality evaluation carried out in [13] shows that the effect of packet losses on depth perception is lower compared to the effect of packet losses on overall image quality. The scheme proposed in this paper therefore allocates more protection for the colour image packets than for the depth map packets in order to improve the quality of received 2-D/3-D video. The protection levels are determined based on the allocated transmission power for colour and depth video data.

The proposed UEP scheme is shown in Fig. 1. The colour plus depth stereoscopic video is coded using a SVC configuration based on the scalable extension of H.264/AVC. The colour and depth video are coded at the base and enhancement layers respectively. This configuration is backward compatible for 2-D video. A performance analysis of this encoding architecture compared to other 3-D coding configurations is provided in [14]. The extractor module separates the colour and depth packets into two packet streams before feeding them into the WiMAX transmission system. The UPA module, shown in Fig. 1, allocates different power levels to the colour and depth bit-streams. The functionality of the UPA module is described in the following section.

Fig. 1. The proposed UEP scheme based on UPA.

### A. Prioritization of 3D Video Packets

The UPA module in Fig. 1 receives channel feedback information from the WiMAX transmission system. It is assumed that the scheme is to operate on pre-encoded video, and therefore the packet loss rate mainly depends upon the transmission rate, transmission power, interference from other users, and losses in the channel. However, power ‘p’ is the dominant factor, and the distortion is denoted by D(r,p), where r denotes the rate information. The problem statement to minimize this distortion due to channel characteristics can be defined as: TeX Source $$\min_{r,p} D(r,p)\quad subject\ to\quad \sum_{k=1}^M p_k \leq P_{ST}\ and\ \sum_{k=1}^M r_k \leq R_{ST}\eqno{\hbox{(1)}}$$where PST and RST are the total power and rate budget for a user, and M is the number of users in the cell.

A suboptimal solution to this problem is proposed below for fast implementation. N numbers of subcarriers need to be distributed among L video layers for m users. The transmitter has information about the channel conditions, which is updated periodically via feedback channels. First, N subchannels are distributed among M users, based upon rate, starting with distribution of equal power to all users, such that: TeX Source $$P_{m,n} = {P_{Total} \over N}\eqno{\hbox{(2)}}$$where Pm,n the power of an mth user's nth subchannel. The rate assigned to a user is given by: TeX Source $$R_m = \sum_{n=1}^N \log_2 \left(1+P_{m,n} {\vert G_{m,n}\vert^2 \over N_o {B\over N}}\right)\eqno{\hbox{(3)}}$$where G2m,n is the nth channel gain of the mth user with a noise density of No. The overall available bandwidth is given by B, and Rm is the transmission rate of the user m. For M users, the optimization of the power can be expressed as: TeX Source $$\max P_{m,n} \sum_{m=1}^M\sum_{n=1}^N \log_2 \left(1+P_{m,n} {\vert G_{m,n}\vert^2 \over N_o {B\over N}}\right)\eqno{\hbox{(4)}}$$The solution of (4) may be found using the methods given in [15]. The first step of the algorithm is the initialization of power levels, and then determination of the number of subchannels for each user in the cell.

• 1-Initialize the set of users, A = {1,2,3,…M}, and Rm = 0

• 2-For m = 1 to M {

• Find n satisfying Gm,n≥Gm,j for all j ∊ A

• Calculate Rm using (3) }

• 3-While A ≠ ⊘

• Find m so that Rm ≤ Ri for all i ≤ M

• 4-For above m, find n satisfying Gm,n ≥ Gm,j for all j ∊ A

• 5-Update Rm using (3), and A = A−{n}

The above algorithm will assign subcarriers to each user in the cell, in proportion with the rate requirement.

### B. Distribution of Subcarriers of a User Among 3-D Video Layers

After all users have been assigned subcarriers, each user has a power budget and a rate budget. Assume Nm,s subchannels are available to the mth user for transmission of data. An extractor in the base station analyzes the importance of the data packet. Colour packets in the base layer are given priority over depth information in the enhancement layer. Based upon feedback about the packet loss ratio, the UPA algorithm calculates power for the subchannels with the help of a lookup table as shown in Table II. The table translates the packet loss rate information into distortion levels, which in turn is used to distribute the power among L video layers for that particular user. In this paper we have used only 2 layers (i.e., colour and depth bit-streams). The objective of the UPA algorithm at this stage is to minimize the received distortion: TeX Source $$\min \quad Distortion\ (p_{m,s})\ subject \ to\ \sum_{i=1}^L P_{m,i} \leq P_{m, S}\eqno{\hbox{(5)}}$$where Pm,S is the total power budget for user m. We assume all subcarriers assigned to a user have equal power. This means allocation of two subcarriers to a particular layer doubles the power, and so forth. Hence, the power increment is available in fixed step increments of ΔPs. The UPA algorithm is provided with information about the number of layers being transmitted to the user, the condition of the subcarrier channels Nm,s, and the number of subcarriers available. The algorithm for distribution of power to video layers is as follows:

• 1-Initialize power for all subchannels Nm,s to Pm,s/Nm,s

• 2-Distribute video layers equally among subchannels Nm,s

• 3-At each feedback interval, distribute the power among layers based upon feedback information of estimated distortion. If DDestimated < Drequired:

where Nm,s,b is the number of subchannels assigned to base layer, and Nm,s,e is the number of subchannels assigned to enhancement layer.

SECTION III

## Experimental Setup and Discussion

This section describes the experimental procedure and the simulation environment for evaluation of the proposed UPA scheme for backward compatible 3-D video.

### A. Parameter Settings

The Orbi and Interview sequences (Resolution: 720 × 576 and frame rate: 25frames/s) are coded offline using the JSVM reference software codec version 8.9. The basic encoding parameters are: IPPP… sequence format, a single reference frame, Content Adaptive Binary Arithmetic Coding (CABAC), and 200 frames. The average encoded bitrate is approximately 1 Mbps for both sequences. The bit-stream extractor module of JSVM is utilized to extract base and enhancement layers and to analyze the transmission bandwidth requirement. The lost frames of each layer are concealed using frame copy method at the JSVM decoder.

A WiMAX baseband error trace simulator is utilized to generate error traces for a range of SNR levels, with the parameters given in Table I, [16]. System level simulations are performed to analyze the effects of unequal power allocation on prioritized colour plus depth 3-D video. This models a time varying channel including the effect of multipath for the ITU Vehicular A scenario. The total received power can be mapped to an SNR value, and the pre-simulated error trace that has the closest SNR is utilized to corrupt the video stream transmitted through the simulator. The power distribution among layers is adjusted based on the feedback about the received video distortion according to lookup tables similar to Table II. These tables are obtained experimentally, and describe the best distribution of power amongst the base and enhancement layers. The simulations are run ten times to obtain stable results.

TABLE I Simulation Parameters
TABLE II Example Lookup Table for MCS5 With ΔPs = 0.55

### B. Results and Discussion

Table III present the performance with and without the proposed UEP scheme for the Orbi and Interview sequences respectively. The colour image sequence achieves improved image quality with the proposed UEP scheme for all the error conditions. This is mainly due to the allocation of more power (more subcarriers) to the colour video packets than depth video packets. Consequently, the colour bit-stream packets have survived under error conditions than depth bit-stream packets. For the proposed UEP scheme, the depth quality degrades rapidly as channel errors increase. In order to evaluate the effect of impaired depth images to the perceived quality subjective evaluation tests are conducted. Two 3-D perceptual attributes namely, overall image quality and depth perception are measured. Figs. 2 and 3 show improved overall image quality with the proposed UEP method. However, the subjective results show no significant effect to depth perception from the resultant depth images with the proposed UEP scheme. Moreover, the depth perception has improved at high error conditions due to the availability of improved texture information.

Fig. 2. Subjective test results for Orbi.
Fig. 3. Subjective test results for Interview.
TABLE III Orbi and Interview With and Without UEP Scheme for mcs5
SECTION IV

## Conclusion

The quality improvements for backward compatible 3-D video services are addressed. Due to the characteristics of colour plus depth video representation, the two video streams (i.e., colour and depth video) have different usage and importance towards the reconstructed content and perceptual quality. As colour image sequences are directly viewed, they are protected ahead of depth image sequences in the proposed UEP method. Moreover, due to the possibility of rendering good quality stereoscopic video even with impaired depth map sequences, less protection has been allocated for associated depth maps. The proposed UEP scheme is implemented using a UPA algorithm which allocates transmission power for colour and depth video sequences based on their distortion estimates. The results show that, the colour image quality achieved superior image quality under a range of channel error conditions. Even though the depth quality is reduced with the proposed scheme, the effect on the perceived quality of reconstructed stereoscopic video is insignificant. Although quality has been optimized in this paper, it should also be possible to optimize the power to achieve energy efficient video transmission.

## Footnotes

Chaminda T.E.R. Hewage, Z. Ahmad, Stewart. T. Worrall, S. Dogan, W.A.C. Fernando, and A. Kondoz are with the I-LAB, Centre for communications Systems Research, University of Surrey, Guildford, GU2 7XH, UK (e.thushara@surrey.ac.uk, zaheer.ahmad@surrey.ac.uk, s.worrall@surrey.ac.uk, s.dogan@surrey.ac.uk, w.fernando@surrey.ac.uk, a.kondoz@surrey.ac.uk)

## References

1. Interactive 3-D TV: Concepts and key technologies

C. Fehn, R. De La Barre, S. Pastoor

Proce. of the IEEE, vol. 94, p. 524–538, 2006

2. Sub-picture: ROI coding and unequal error protection

M. M. Hannuksela, Y.-K. Wang, M. Gabbouj

Proceedings of International Conference on Image Processing, 2002-06, 537–540

3. Joint media-channel aware unequal error protection for wireless scalable video streaming

H. Mansour, P. Nasiopoulos, V. Krishnamurthy

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2008), 2008-03, 1129–1132

4. A novel adaptive unequal error protection method for scalable video over wireless networks

A. Naghdinezhad, M. R. Hashemi, O. Fatemi

IEEE Inter. Symp. on Consu. Electro., 2007-06, 1–6

5. Rate-distortion optimized layered stereoscopic video streaming with raptor codes

T. A. Serdar, A. Aksay, C. Bilen, G. B. Akar, E. Arikan

Packet Video, p. 98–104, 2007-11

6. An unequal error protection method for progressively transmitted 3D models

G. AlRegib, Y. Altunbasak, J. Rossignac

IEEE Transactions on Multimedia, vol. 7, issue (4), p. 766–776, 2005-08

7. Frame concealment algorithm for stereoscopic video using motion vector sharing

C. T. E. R. Hewage, S. Worrall, S. Dogan, A. M. Kondoz

Hannover, Germany
Proceedings of IEEE International Conference on Multimedia & Expo 2008 (ICME2008), 2008-06

8. Unequal error protection by modulation with unequal power allocation

T. Bruggen, P. Vary

IEEE Communications Letters, vol. 9, issue (6), p. 484–486, 2005-06

9. Multi-view video plus depth representation and coding

P. Merkle, A. Smolic, K. Muller, T. Wiegand

San Antonio, TX
IEEE International Conf. on Image Processing (ICIP 2007), 2007-10, 201–204

10. White Paper on ISO/IEC 23002-3 Auxiliary Video Data Representation

A. Bourge, C. Fehn

2006, ISO/IEC JTC1/SC29/WG11 N8039

11. A 3D-TV approach using depth-image-based rendering (DIBR)

C. Fehn

Proc. of Visualization, Imaging, and Image Processing (VIIP'03), 2003, 482–487

12. Stereoscopic TV over IP

C. T. E. R. Hewage, S. Worrall, S. Dogan, H. K. Arachchi, A. M. Kondoz

London, UK
Proceedings of the 4th IET European Conference on Visual Media Production (CVMP'2007), 2007-11

13. Quality evaluation of colour plus depth map based stereoscopic video

C. T. E. R. Hewage, S. Worrall, S. Dogan, A. M. Kondoz

IEEE Journal of Selected Topics in Signal Processing, 2008-05, accepted to, (accepted for publication in 2009-04)

14. Comparison of stereo video coding support in MPEG-4 MAC, H.264/AVC and H.264/SVC

C. T. E. R. Hewage, H. A. Karim, S. Worrall, S. Dogan, A. M. Kondoz

Proc. of IET Visual Information Engineering-VIE07, 2007

15. Adaptive resource allocation in multiuser OFDM systems with proportional rate constraints

S. Zukang, J. G. Andrews, B. L. Evans

IEEE Transactions on Wireless Communications, vol. 4, issue (6), p. 2726–2737, 2005

16. SUIT Deliverable Document 208

2007-01, WIMAX Modeling, FP6 reference number IST-4-028042

## Cited By

No Citations Available

## Keywords

### INSPEC: Non-Controlled Indexing

No Keywords Available

### Authors Keywords

No Keywords Available

### More Keywords

No Keywords Available

No Corrections

## Media

No Content Available
This paper appears in:
International Symposium on Circuits and Systems
Issue Date:
2009
On page(s):
125 - 128
ISBN:
N/A
Print ISBN:
978-1-4244-3827-3
INSPEC Accession Number:
10760362
Digital Object Identifier:
10.1109/ISCAS.2009.5117701
Date of Current Version:
26 Jun, 2009