Scheduled System Maintenance:
Some services will be unavailable Sunday, March 29th through Monday, March 30th. We apologize for the inconvenience.
By Topic

The empirical distribution of good codes

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

The purchase and pricing options are temporarily unavailable. Please try again later.
2 Author(s)
Shamai, S. ; Dept. of Electr. Eng., Technion-Israel Inst. of Technol., Haifa, Israel ; Verdu, S.

Let the kth-order empirical distribution of a code be defined as the proportion of k-strings anywhere in the codebook equal to every given k-string. We show that for any fixed k, the kth-order empirical distribution of any good code (i.e., a code approaching capacity with vanishing probability of error) converges in the sense of divergence to the set of input distributions that maximize the input/output mutual information of k channel uses. This statement is proved for discrete memoryless channels as well as a large class of channels with memory. If k grows logarithmically (or faster) with blocklength, the result no longer holds for certain good codes, whereas for other good codes, the result can be shown for k growing as fast as a certain fraction of blocklength

Published in:

Information Theory, IEEE Transactions on  (Volume:43 ,  Issue: 3 )