Open-Source MQTT Evaluation

MQTT is a lightweight publish-subscribe protocol used in the Internet of Things. Its popularity leads to several implementations in different languages. In this paper, we evaluate the most popular open-source MQTT implementations, Mosquitto, HiveMQ, EMQX, VerneMQ, MQTT.js and Paho. Our evaluation includes interoperability, resource consumption and latency. We create a generic test framework independent of any MQTT implementation or language. According to our interoperability results, major client and server implementations conform to the base requirements of the standard and thus can interoperate. The language of implementation is a crucial factor for resource consumption and causes considerable differences in scalability. Our results show that latencies between implementations in lossy networks differ. Overall, there is a trade-off between resource consumption and network latency.