Dynamically instrumenting message-passing programs using virtual clocks

Analysing and visualising the run time behaviour of a distributed program requires collecting performance data during its execution. This is usually done through program instrumentation that inserts trace recording instructions into the program. The intrusiveness of inserted extra code to the original program may make the existing errors vanish, or can cause new errors to appear. It will not only change the real time execution speed, but also change the probability of making particular non deterministic choices. W. Cai and S.J. Turner (1994) proposed a logical clock approach (LCA) to control the inter process communication. It uses a logical clock in each process to reflect the real time execution of the process when running without monitoring. Not relying on any special hardware, this approach achieves minimum intrusiveness and makes the monitoring effect on the original program completely independent from the amount of time spent on monitoring activities. A main problem with LCA is that in an event of non deterministic communication and when several processes wait on each other's logical clock to advance, a deadlock may occur. LCA uses a carrier null message algorithm to break the deadlock situation. But sending an extra null message will distort the event timing information. Therefore, LCA is only suitable for debugging rather than for performance tuning. We present a major improvement to LCA, which preserves the timing information without introducing any deadlock. Our improved approach can support both debugging and performance tuning. We call our clocks virtual clocks to avoid confusion.