By Topic

Mutual Information, Relative Entropy, and Estimation in the Poisson Channel

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Rami Atar ; Technion—Israel Institute of Technology, Haifa, Israel ; Tsachy Weissman

Let be a nonnegative random variable and let the conditional distribution of a random variable , given , be Poisson , for a parameter . We identify a natural loss function such that: (1) the derivative of the mutual information between and with respect to is equal to the minimum mean loss in estimating based on , regardless of the distribution of ; (2) when is estimated based on by a mismatched estimator that would have minimized the expected loss had , the integral over all values of of the excess mean loss is equal to the relative entropy between and . For a continuous time setting where is a nonnegative stochastic process and the conditional law of , given , is that of a non-homogeneous Poisson process with intensity function , under the same loss function: (1) the minimum mean loss in causal filtering when is equal to the expected value of the minimum mean loss in noncausal filtering (smoothing) achieved with a channel whose parameter is uniformly distributed between 0 and . Bridging the two quantities is the mutual information between and ; (2) this relationship between the mean losses in causal and noncausal filtering holds also in the case where the filters employed are mismatched, i.e., optimized assuming a law on which is not the true one. Bridging the two quantities in this case is the sum of the mutual information and the relative entropy between the true and the mismatched distribution of . Thus, relative entropy quantifies the excess estimation loss due to mismatch in this setting. These results are parallel to those recently found for the Gaussian channel: the I-MMSE relationship of Guo , the relative entropy and mismatched estimation relationship of Verdú, and the relationship between causal and noncasual mismatched estimation of Weissman.

Published in:

IEEE Transactions on Information Theory  (Volume:58 ,  Issue: 3 )