By Topic

The minimax distortion redundancy in noisy source coding

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Dembo, A. ; Dept. of Stat., Stanford Univ., CA, USA ; Weissman, T.

Consider the problem of finite-rate filtering of a discrete memoryless process {Xi}i≥1 based on its noisy observation sequence {Zi}i≥1, which is the output of a discrete memoryless channel (DMC) whose input is {Xi}i≥1. When the distribution of the pairs (Xi,Zi), PX,Z, is known, and for a given distortion measure, the solution to this problem is well known to be given by classical rate-distortion theory upon the introduction of a modified distortion measure. We address the case where PX,Z, rather than being completely specified, is only known to belong to some set Λ. For a fixed encoding rate R, we look at the worst case, over all θ∈Λ, of the difference between the expected distortion of a given scheme which is not allowed to depend on the active source θ∈Λ and the value of the distortion-rate function at R corresponding to the noisy source θ. We study the minimum attainable value achievable by any scheme operating at rate R for this worst case quantity, denoted by D(Λ, R). Linking this problem and that of source coding under several distortion measures, we prove a coding theorem for the latter problem and apply it to characterize D(Λ, R) for the case where all members of Λ share the same noisy marginal. For the case of a general Λ, we obtain a single-letter characterization of D(Λ, R) for the finite-alphabet case. This gives, in particular, a necessary and sufficient condition on the set Λ for the existence of a coding scheme which is universally optimal for all members of Λ and characterizes the approximation-estimation tradeoff for statistical modeling of noisy source coding problems. Finally, we obtain D(Λ, R) in closed form for cases where Λ consists of distributions on the (channel) input-output pair of a Bernoulli source corrupted by a binary-symmetric channel (BSC). In particular, for the case where Λ consists of two sources: the all-zero source corrupted by a BSC with crossover probability r and the Bernoulli(r) source with a noise-free channel; we find that universality be- comes increasingly hard with increasing rate.

Published in:

Information Theory, IEEE Transactions on  (Volume:49 ,  Issue: 11 )