A semi-adaptive arithmetic coding scheme for Chinese textual data

This paper presents a compression scheme for Chinese text files. Due to the skewness of the distribution of Chinese ideograms the arithmetic coding method is adopted. To reduce the overhead incurred by the frequency table in the compressed output due to the large number of Chinese ideograms, differential coding and arithmetic coding are used to produce a two-level storage structure for the frequency table. Evaluations of the proposed algorithm against several popular compression schemes show that the compression efficiency is significantly improved. This algorithm should also be applicable to other ideogram-based or oriental language texts.