Sidney and RDS: an evaluation of two persistent storage systems

Comparative experimentation is increasingly important in computer science, but performing such experiments can be challenging. The paper presents a set of experiments that compare the performance of two persistent storage managers, and answer the question of whether the safer storage manager has performance comparable to the less safe one. This comparison was difficult for a number of reasons, among them: relatively few programs using either storage manager existed and no established benchmarks existed, and the two techniques are incompatible at the source code level, thus making a direct comparison impossible. In particular one storage manager used a malloc-and-free style of dynamic storage allocation, while the other used a high performance concurrent garbage collector. A number of approaches were used to overcome this difficulty. The most novel approach involved tracing the memory management of a production program that used the malloc-and-free based storage manager and then replaying the trace in an environment that allowed garbage collection and malloc-and-free to be compared. The study represents the most extensive study of a garbage collected persistent storage system to date.