Cart (Loading....) | Create Account
Close category search window

Universally attainable error exponents for rate-distortion coding of noisy sources

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

1 Author(s)
Weissman, T. ; Dept. of Electr. Eng., Stanford Univ., CA, USA

Consider the problem of rate-constrained reconstruction of a finite-alphabet discrete memoryless signal Xn=(X1,...,Xn), based on a noise-corrupted observation sequence Zn, which is the finite-alphabet output of a discrete memoryless channel (DMC) whose input is Xn. Suppose that there is some uncertainty in the source distribution, in the channel characteristics, or in both. Equivalently, suppose that the distribution of the pairs (Xi,Zi), rather than completely being known, is only known to belong to a set Θ. Suppose further that the relevant performance criterion is the probability of excess distortion, i.e., letting Xˆn(Zn) denote the reconstruction, we are interested in the behavior of Pθ(ρ(Xn,Xˆn(Zn))>dθ), where ρ is a (normalized) block distortion induced by a single-letter distortion measure and Pθ denotes the probability measure corresponding to the case where (Xi,Zi)∼θ, θ∈Θ. Since typically this probability will either not decay at all or do so at an exponential rate, it is the rate of this decay which we focus on. More concretely, for a given rate R ≥ 0 and a family of distortion levels {dθ}θ∈Θ, we are interested in families of exponential levels {Iθ}θ∈Θ which are achievable in the sense that for large n there exist rate-R schemes satisfying -1/nlog Pθ (ρ(Xn, Xˆn(Zn)) > dθ) ≥ Iθ, for all θ ∈ Θ. Our main result is a complete "single-letter" characterization of achievable levels {Iθ}θ∈Θ per any given triple (Θ,R,{dθ}θ∈Θ). Equipped with this result, we later turn to addressing the question of the "right" choice of {Iθ}θ∈Θ. Relying on methodology recently put forth by Feder and Merhav in the context of the composite hypothesis testing problem, we propose a- competitive minimax approach for the choice of these levels and apply our main result for characterizing the associated key quantities. Subsequently, we apply the main result to characterize optimal performance in a Neyman-Pearson-like setting, where there are two possible noise-corrupted signals. In this problem, the goal of the observer of the noisy signal, rather than having to determine which of the two it is (as in the hypothesis testing problem), is to reproduce the underlying clean signal with as high a fidelity as possible (e.g., lowest number of symbol errors when distortion measure is Hamming), under the assumption that one source is active, while operating at a limited information rate R and subject to a constraint on the fidelity of reconstruction when the other source is active. Finally, we apply our result to characterize a sufficient condition for the source class Θ to be universally encodable in the sense of the existence of schemes attaining the optimal distribution-dependent exponent, simultaneously for all sources in the class. This condition was shown in an earlier work to suffice for universality in expectation.

Published in:

Information Theory, IEEE Transactions on  (Volume:50 ,  Issue: 6 )

Date of Publication:

June 2004

Need Help?

IEEE Advancing Technology for Humanity About IEEE Xplore | Contact | Help | Terms of Use | Nondiscrimination Policy | Site Map | Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest professional association for the advancement of technology.
© Copyright 2014 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.