Bounds on Codeword Lengths of Optimal Codes for Noiseless Channels

For the case in which the code alphabet consists of M letters, we derive the optimum probability distribution of a discrete source that assures the minimum average codeword length. For the case of Huffman encoding, we derive the upper and lower bounds of average codeword length.