By Topic

A new outlook on Shannon's information measures

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

1 Author(s)
Yeung, R.W. ; AT&T Bell Lab., Holmdel, NJ, USA

The author presents a new approach to understanding the underlying mathematical structure of Shannon's information measures, which provides answers to the following two questions for any finite number of random variables. (1) For any information-theoretic identity, is there a corresponding set-theoretic identity via the formal substitution of symbols? (2) For any set-theoretic identity, is there a corresponding information-theoretic identity and, if so, in what sense? The author establishes the analogy between information theory and set theory. Therefore, each information-theoretic operation can formally be viewed as a set-theoretic operation and vice versa. This point of view, which the author believes is of fundamental importance has apparently been overlooked in the past by information theorists. As a consequence the I-diagram, which is a geometrical representation of the relationship among the information measures, is introduced. The I-diagram is analogous to the Venn diagram in set theory. The use of the I-diagram is discussed.

Published in:

Information Theory, IEEE Transactions on  (Volume:37 ,  Issue: 3 )