A Workbench for Quantitative Comparison of Databases in Multi-Robot Applications

Robots generate large amounts of data which need to be stored in a meaningful way such that they can be used and interpreted later. Such data can be written into log files, but these files lack the querying features and scaling capabilities of modern databases - especially when dealing with multi-robot systems, where the trade-off between availability and consistency has to be resolved. However, there is a plethora of existing databases, each with its own set of features, but none designed with robotic use cases in mind. This work presents three main contributions: (a) structures for benchmarking scenarios with a focus on networked multi-robot architectures, (b) an extensible workbench for benchmarking databases for different scenarios that makes use of Docker containers and (c) a comparison of existing databases given a set of multi-robot use cases to showcase the usage of the framework. The comparison gives indications for choosing an appropriate database.