Life Cycle of Transactional Data in In-memory Databases

The last decade has witnessed an explosive growth in database engines optimized for main memory based execution. However, the requirement to store all the data in-memory makes such processing commercially costly and unviable for very large databases. In this paper, we present novel techniques optimized for transactional workloads where only a small portion of the entire database is "hot" and needs to be in-memory. The SAP ASE hybrid database engine delivers high-performance transaction processing transparently on tiered storage consisting of traditional page-oriented disk based storage, for cold data, and in-memory row storage for fast processing. Our techniques make storage choice for the rows based on access patterns of the OLTP workload. This is done by monitoring and analysing the workload running in the system with minimal impact to the transaction performance. The techniques also adapt to the workload to alter storage choices for the data, which is completely transparent to the application and provides continuous data access. The storage choices are not made at gross table or partition level but made at the level of individual row and type of DML operation. Our result show that these techniques help reduce the memory footprint by a large margin in OLTP workload even while providing performance parity with a setup where all data is stored in-memory.