Lightweight Operating System Services for Incremental Checkpointing in Speculative Discrete Event Simulation on Linux Platforms

One way for supporting incremental checkpointing is the exploitation of classical memory protection services—in particular the mprotect (…) system call offered by Posix compliant operating systems—for intercepting memory-writes and identifying dirty pages in the address space. However, this solution involves Inter-Processor-Interrupt (IPI) and the associated handling mechanisms, which show costs that increase when scaling up the level of parallelism in the underlying hardware architecture. For HPC contexts like speculative (aka optimistic) parallel discrete event simulation, where checkpointing and state restore massively take place in order to maintain causality among the concurrent simulation objects, these costs may impact performance in a non-negligible manner. In this work, we present the design of operating system services that enable write-protection and the tracing of dirtied pages via per CPU setup of the MMU (Memory Management Unit), completely avoiding the usage of IPI and their management. Hence, we provide a solution where the cost for setting up the incremental checkpointing support of a simulation object processed by a specific thread at a given time is definitely limited, compared to the aforementioned classical case. Our design has been devised for Linux, although it can be ported to other operating systems, and has been integrated for its testing in the USE (Ultimate Share Everything) open source parallel simulation environment.