Scalable Index Update for Block-Level Continuous Data Protection

A block-level continuous data protection (CDP) system logs every disk update to a network storage server it protects, so as to support more flexible recovery time objective (RTO) and recovery point objective (RPO). To provide efficient access to historical snapshots, block-level CDP systems maintain multiple index structures, each of which needs to be updated whenever a disk block update operation is logged. Because these index structures are too large to be held in memory, updating their on-disk versions in real time becomes a major performance bottleneck that prevents existing CDP systems from scaling to large data backup applications. This paper describes the design and implementation of a high-performance index update mechanism that logs index updates, batches them in memory, and commits them using mostly sequential disk I/O. Sequential commit greatly reduces the cost of bringing in and writing back each on-disk index page. Update batching further amortizes this cost over multiple index update operations. Empirical performance measurements demonstrate that the proposed technique improves the index update throughput by more than an order of magnitude and reduces the performance overhead associated with index updates from 95% to under 15%.