CODIS: A New Compression Scheme for Bitmap Indexes

Bitmap indexing is a promising approach for indexing. However the huge space consumption hinders the wide adoption of bitmap indexing, especially in memory-critical area such as packet classification. To this end, a variety of compression scheme are proposed to reduce the space consumption and simultaneously maintain the fast calculation which is a focused feature of bitmap indexing. In this paper, a novel compression scheme, named COmpressing DIrty Snippet (CODIS), is proposed. It is based on the Word-Aligned Hybrid(WAH) algorithm. The basic idea is to trade some payload for flexibility so that the probability of space reduction is raised, which achieves better compression. CODIS is verified by experiments on part of the network traffic data from CAIDA 2016. The results show that, comparing to WAH, CODIS increases the time for intersection at a rate of about 7% while reduces 39% of the space consumption. Comparing to COMPAX, it takes 11% more space but reduces 19% of time for intersection. Comparing to PLWAH, it is better in both space consumption and inter-bitmap operations. CODIS takes the least time to decode bitmap indexes into integers.