Proposal for a Flexible Benchmark for Agent Based Models

As hardware capabilities and scientific understanding of social and biological systems improves, researchers are able to model complex system with increasing detail. This change increases the demand on Agent Based Modeling (ABM) software, making the performance of the software more critical. For researchers to choose the best system to implement their models, they must understand not only the capabilities of the platforms, which is available in the documentation, but also the relative performance to other software options. Currently, there is a dearth of information on the performance of ABM platforms and no standardized way to compare the performance of different platforms. To address this gap, the proposed NothingModel benchmark is a flexible benchmark to reflect a wide variety of ABMs and allow the developers of models and ABM platforms to compare platforms under a range of conditions. Set up as a series of building blocks, it allows the users to vary the scale, memory use, communication, computation, and heterogeneity of a sample model, implemented with multiple tools. Along with the written description of the model, we include a reference implementation in NetLogo to demonstrate the specifics and provide comparison to other tools.