Improving LZW Compression of Unicode Arabic Text Using Multi-Level Encoding and a Variable-Length Phrase Code

This paper introduces a novel approach to enhance the efficiency of compressing Arabic Unicode text using the Lempel-Ziv-Welch (LZW) technique. This method includes two stages: transformation and compression. During the first phase, a multi-level scheme that works according to the level of words, syllables, and characters replaces multi-byte symbols with single-byte symbols, resulting in a binary output of 51%-75% smaller than the actual size and effective for compression. In the second phase: the outputs of the previous phase are received as inputs to the adaptive LZW technique, attached to a value representing the length of the initial phrase (minimum code length). This value is automatically determined according to the size of the data source to enhance the performance of LZW. The original data size is included in the compressed file to be used during the decompression process to detect the length of the initial phrase. The compression ratio achieved by the proposed method was compared to that of the traditional LZW technology that uses multi-byte encoded characters and a fixed initial length phrase, as well as two recent technologies, DEFLATE and Gzip. Experimental results indicate that our method achieves an average compression rate of about 71% and outperforms other methods for all forms of Arabic texts, with improved LZW able to compress an additional 7% or more or less of the files it compresses. Variable-length dictionary LZW continuously displays a significant difference in compression ratios for small files compared to modern methods, whether it uses a variable-width- encoding scheme only or with multi-level encoding as a precompression step. Multi-level schema can be used as a preprocess to other compression techniques, especially those that work efficiently with binary data. Also, the original data volume can be used as a private key within data security and encryption applications.