Measurement of Information of a Discrete Source and Channel Capacity

Summary

The purpose of a message is to provide information to a recipient. Information can be transported (transmitted), stored (memorized) and transformed (processed). This chapter presents some examples of discrete sources, including memoryless discrete source, discrete source with memory, ergodic source, and first order Markovian source. The ergodicity of an information source implies that a temporal average calculated over an infinite duration is equal to the statistical average. Between the source of information and the destination, there is the medium through which information is transmitted. This medium, including the equipment necessary for transmission, is called the transmission channel. Claude Shannon introduced the concept of channel capacity, to measure the efficiency with which information is transmitted, and to find its upper limit. The chapter also illustrates Shannon's theorem in terms of capacity of a communication system.