By Topic

Reconstruction sequences and equipartition measures: an examination of the asymptotic equipartition property

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

4 Author(s)
J. T. Lewis ; Dublin Inst. for Adv. Studies, Ireland ; C. E. Pfister ; R. P. Russell ; W. G. Sullivan

We consider a stationary source emitting letters from a finite alphabet A. The source is described by a stationary probability measure α on the space Ω:=AIN of sequences of letters. Denote by Ωn the set of words of length n and by αn the probability measure induced on Ωn by α. We consider sequences {Γn⊂Ωn: n∈IN} having special properties. Call {Γn⊂Ωn: n∈IN} a supporting sequence for α if limn α nn]=1. It is well known that the exponential growth rate of a supporting sequence is bounded below by h Sh(α), the Shannon entropy of the source α. For efficient simulation, we require Γn to be as large as possible, subject to the condition that the measure αn is approximated by the equipartition measure βn[·|Γn], the probability measure on Ωn which gives equal weight to the words in Γn and zero weight to words outside it. We say that a sequence {Γn⊂Ωn: n∈IN} is a reconstruction sequence for α if each Γn is invariant under cyclic permutations and limn βn [·|Γn]=αm for each m∈IN. We prove that the exponential growth rate of a reconstruction sequence is bounded above by hSh(α). We use a large-deviation property of the cyclic empirical measure to give a constructive proof of an existence theorem: if α is a stationary source, then there exists a reconstruction sequence for α having maximal exponential growth rate; if α is ergodic, then the reconstruction sequence may be chosen so as to be supporting for α. We prove also a characterization of ergodic measures which appears to be new

Published in:

IEEE Transactions on Information Theory  (Volume:43 ,  Issue: 6 )