Transforming Big Data: A Novel Arithmetic Compression Method Based on Symbol Frequency

The article proposes a new method for arithmetic compression of large volumes of data. The used compression approach is based on dividing the original data into slices depending on the frequency of occurrence of symbols. The resulting cuts are subjected to arithmetic compression, which ensures an effective reduction of the volume of data, while maintaining the quality and reliability of information. The new method studies the statistical information on the frequency of occurrence of characters in the data set, which allowed setting the optimal place for dividing the data into slices. The main advantage of the proposed method lies in its ability to adapt to the specific characteristics of the data set, providing optimal partitioning and compression. This makes it possible to achieve a higher degree of stability with traditional arithmetic embossing methods while maintaining data quality. An additional advantage is the possibility of applying the method to various information types of data, which makes it a universal tool for optimizing the storage and transmission of large volumes.