A data streaming performance evaluation using resource constrained edge device

High-level automation comes as standard in the Industrial IoT(IIoT) environment. Automated IIoT devices lead the improvement of factory operational efficiency. They helped to reduce operational time and personnel expenses, and helped to increase operational accuracy. IIoT devices for smart factory are various in types, and they are designed to fulfill their designated works for factory in diverse ways. Their jobs include prediction of operation failure and anomaly detection to avert factory downtime. If stream data, which is generated by machines in the factory, can be processed faster than before, the downtime could be shorter and prediction could be made earlier. To make stream data processing time shorter, performing it close to the machine could be a good approach. If an IIoT device is smart enough to process data, it can obtain meaningful information by processing data by itself. We can also use another machine to collect data from small sensors, and process the data to make decision. We address this type of data processing which is performed at the machines as ”Edge Computing”. ”Edge” originally meant the edge of the network. If data processing can be performed without being transmitted to the cloud or another storage, it would be done faster. ”Edge” devices usually have lower hardware capacity than ”Server”, so we wanted to know which data processing engine can be used for edge devices. We selected three representative engines: Apache Flink, Apache Storm and Apache Spark streaming. We installed them on the Raspberry Pi 3 and evaluated their latencies and throughputs. The conclusion is that Apache Flink and Apache Spark Streaming showed competing results in all tests while Apache Storm showed worse performance than the others. It required more memory resources to excute its basic components, and this caused its long warming up time to perform our simple algorithms.