Tackling the Memory Balancing Problem for Large-Scale Network Simulation

A key obstacle to large-scale network simulation over PC clusters is the memory balancing problem where a memory-overloaded machine can slow down an entire simulation due to disk I/O overhead. Memory balancing is complicated by (i) the dfficulty of estimating the peak memory consumption of a group of nodes during network partitioning-a consequence of per-node peak memory not being synchronized-and (ii) trade-off with CPU balancing whose cost metric depends on total-as opposed to maximum-number of messages processed over time. We investigate memory balancing for large-scale network simulation which admits solutions for memory estimation and balancing not availed to small-scale or discrete-event simulation in general. First, we advance a measurement methodology for accurate and efficient memory estimation, and we establish a trade-off between memory and CPU balancing under maximum and total cost metrics. Second, we show that joint memory-CPU balancing can overcome the performance trade-off-in general not feasible due to constraint conflicts-which stems from network simulation having a tendency to induce correlation between maximum and total cost metrics. Performance evaluation is carnied out using benchmark applications with varying traffic characteristics-BGP routing, worm propagation under local and global scanning, and distributed client/server system-on a testbed of 32 Intel times86 machines running a measurement-enhanced DaSSF.