LCTL: Lightweight Compression Template Library

For fast and efficient data processing, a common approach in many application domains is to mainly store and process data in form of arrays of integers offering several benefits. For example, with the help of some additional lightweight computations for lossless integer compression, the necessary memory space can be dramatically reduced. Moreover, compressed integer values offer advantages for data processing such as increasing the effective bandwidth to reduce the memory wall effect. Thus, a large corpus of lightweight integer compression formats has been developed. Unfortunately, there is no approach available that allows to define a specific format in a consistent way and to generate executable code for compression and decompression from it. To overcome that, we present the Lightweight Compression Template Library (LCTL) in this paper. As we are going to show, LCTL allows (i) the implementation of a variety of compression formats in an abstract way, (ii) the generation of efficient executable code for compression, and (iii) the automatic derivation of the decompression routines out of a compression format. Thus, LCTL offers a unique approach to comprehensively and systematically implement the large corpus of lightweight integer compression formats.