A 2-cycle algorithm for source coding with a fidelity criterion

Since the birth of rate-distortion theory with the landmark paper of Shannon, research has centered on the extension of his theory to new situations and on the calculation of rate-distortion functions. Comparatively little has been done toward developing efficient algorithms for encoding information sources at rates near Shannon's rate-distortion limit. This paper reports one such algorithm, the 2-cycle algorithm, used with randomly chosen tree codes. In analyzing the 2-cycle algorithm, we present the first theoretical analysis of a realizable algorithm for tree coding of sources with a fidelity criterion. The algorithm may prove to be the first practical method of coding whose design actually derives from rate-distortion theory. Analysis proceeds by bounding methods related to difference equations and branching processes. Upper bounds on average distortion and encoding work are obtained and the latter are shown bounded as long as the coding rate exceeds Shannon's rate-distortion function. Of particular interest is the stack-searched branching process bound of Section IV. The two cycles of the algorithm are described fully in terms of expectations and probabilities, and recursions are listed to compute these quantities. Concluding the paper is numerical analysis, by theory and by simulation, with respect to the binary i.i.d, source and Hamming distortion measure. These numerical results are sufficient to optimize the algorithm over its free parameters.