Causally ordered multicast: the conservative approach

Process group toolkits provide methods to structure a system as a set of groups of cooperating processes, to detect process failures, and to order events (by ordering messages). Such tools have a performance cost for applications, particularly when a system is built using a large number of overlapping groups. We built an event-driven simulation to study performance of causally ordered message delivery in large systems composed of overlapping groups. Our studies, the first ever of multiple group systems, reveal some conditions under which the delays can be very large: two orders of magnitude greater than when delays are not required. Further, in a large system these delays can lead to increased system burstiness which limits system scalability. These results suggest that a system supporting multiple overlapping groups needs to be carefully designed and the system should often provide users with control over when to apply ordering guarantees.