Re-execution of distributed programs to detect bugs hidden by racing messages

Finding errors in non-deterministic programs is complicated by the fact that an anomaly may occur during one program execution and not the next. Our objective is to provide a practical, yet powerful testing environment for distributed systems, using re-execution. We focus on re-executing the program under a strictly different message ordering. We show that messages are grouped into waves, such that any two messages from different waves must always be received in the same order. We provide an algorithm that produces a re-execution that maximizes the number of reordered pairs of message delivery events. We also provide an efficient online algorithm for detecting racing messages.