Asymptotic Equipartition Property and Channel Capacity

Two of the main theorems in information theory are the source coding theorem and the channel coding theorem. This chapter introduces the asymptotic equipartition property (AEP), a mathematical tool based on the law of large number. Based on the property, the source coding theorem is presented. The chapter presents the channel coding theorem by applying the property to the relation between the transmitted and received symbols in a communication system. There are of course numerous types of channels, and they can be made arbitrarily complicated depending on the level of the modeling. The chapter concentrates on discrete memoryless channels (DMC). It describes one of the most well‐known channels, the binary symmetric channel (BSC). The chapter introduces Shannon's channel coding theorem, which relates the information channel capacity to the coding rate. For this purpose, the AEP and typical sequences need to be extended as jointly typical sequences.