Text preprocessing using Enhanced Intelligent Dictionary Based Encoding (EIDBE)

Intelligent Dictionary Based Encoding (IDBE), an encoding strategy offers higher compression ratios and rate of compression. Transforming text into some intermediate form by using IDBE is the basic philosophy of this compression technique. It is observed that a better compression is achieved by using IDBE as the preprocessing stage for the BWT based compressor. This paper aims at developing such a new and advanced technique called "Enhanced Intelligent Dictionary Based Encoding (EIDBE)". This approach is used in conjunction with BWT and is a dictionary based lossless reversible transformation that can be applied to a source text to improve existing or backend algorithm's ability to compress. EIDBE achieves better compression at the preprocessing stage and enough redundancy is retained for the compression algorithms to get better results. The experimental results of this compression method are also analysed.