Efficient Integer Coding for Arbitrary Probability Distributions

Performance bounds on the average codeword length of Golomb codes for arbitrary probability distributions are derived in terms of the mean. Then based on Golomb codes, a class of extended gamma codes which are the generalizations of Elias gamma code is constructed and proved to be universal. Optimal decision rules for choosing parameters of Golomb codes and extended gamma codes for arbitrary probability distributions are also derived. Finally, the concept of maximum entropy codes is introduced and the existence of such codes is investigated for source classes with linear constraints. Golomb codes with optimal parameters turn out to be maximum entropy codes for sources with a fixed mean.