coreemu-lab: An Automated Network Emulation and Evaluation Environment

Developing new solutions for challenging communication environments requires extensive testing in various different scenarios. While discrete-event network simulators scale well, their main use is in evaluating specific algorithms and not necessarily assessing the real-world performance of an actual application. In contrast, full network emulation is a costly but more realistic way to perform an evaluation. These types of evaluation provide more valuable insights for both developers and users. Nevertheless, oftentimes the wheel is reinvented to provide a newly written simulation environment for a specific networking software or handwritten evaluation and reporting scripts for common metrics. Unfortunately, the downside of customized solutions is potential oversights regarding important metrics, and later finding out various problems when deploying the software in the field. In this paper, we present coreemu-lab, a novel framework that automates the process of orchestrating different monitoring services in an emulated, lightweight networking environment. This includes simulated mobility, automated data collection, and analysis of common metrics. It is specifically designed for the evaluation of resilient, decentralized communication software in challenging, mobile scenarios. Our evaluation shows the flexibility and ease of use of our proposed multi-platform, open-source solution.