A Unified Benchmarking Framework for Evaluating Discrete Event Simulation Engines

Benchmarks are essential tools in computer science for measuring hardware and software performance. In the Discrete-Event System (DES) community, improving simulation performance is crucial. However, comparing different DES simulators is difficult due to the lack of standardized metrics, complicating direct comparisons. A well-defined benchmark suite would help researchers evaluate simulation engine improvements. This paper proposes a set of metrics for assessing DES performance across sequential, parallel, and distributed architectures. Our approach is based on the Discrete Event System Specification (DEVS) formalism, defining parameters such as model structure and event count, with a focus on total simulation time. This framework provides a consistent method for evaluating DES performance, facilitating direct simulator comparisons. We tested the benchmark suite on various discrete event simulators, including both DEVS-based and non-DEVS-based engines, demonstrating its versatility and effectiveness in assessing performance across different platforms. Our results highlight the benchmark’s ability to improve simulation evaluation.