Real-World Evaluation of Performance and Stability of a Smart-Home System with Mesh Network Devices

In a smart-home system, a low latency response is the key factor in providing a smooth user experience. Smart-home devices that communicate over Wi-Fi can simplify installation and reduce costs, but the communication channel introduces additional uncertainty to the system. Ensuring a short response time is one of the main considerations, when designing such devices. This paper focuses on the testing of the response time performance and the long-term stability of a commercial smart-home system, which features individual nodes connected over a mesh Wi-Fi network and wired switches connected to the main controller board via a CAN connection. Three aspects are evaluated: The effect of the mesh network structure and the location of the MQTT broker on the response times of individual nodes, the response time of outputs triggered by wired switches, and the long-term stability of both wired and wireless device response times. The testing is conducted using custom Python scripts which send the ping command and measure the delay. Testing of wired switch responses is facilitated by an external microcontroller. The results show a clear indication that additional mesh hops and a cloud based MQTT broker introduce a delay in the response, but it is short enough to not negatively affect the user experience. The testing of the output response time revealed a flaw in the controller, which could negatively affect the user experience when switching many outputs at once. The overall stability of the system was good.