Comparative Performance Analysis of Edge-AI Devices in Deep Learning Applications

The fusion of edge computing and artificial intelligence, known as Edge AI, represents a paradigm shift that facilitates the direct execution of AI algorithms on edge devices. As these devices become increasingly powerful, their role in developing and deploying AI systems becomes more significant. By eliminating the need to transmit and analyze data at remote machines, Edge AI applications can significantly reduce latency and enhance efficiency by processing data closer to the source. In this study, we thoroughly investigate the performance of our object classification model deployed in a vision inspection system on four types of edge devices (Jetson AGX Orin, Jetson Orin Nano, NUC, and Raspberry Pi). Our object classification models are trained using proprietary industrial datasets provided by industry partners. These models, in FP32, are converted into lower precision processing, being INT8, to evaluate the accuracy variation between FP32 and INT8 precision, and inference speed for different edge devices. In our experiments, we identified that the average accuracy deviation for INT8 models is −2.78%, with some models exhibiting variations exceeding −10.95%. Most devices have an average inference speed less than 100 ms per image (as requested by industrial partners), except the Raspberry Pi, which records more than 2 seconds of inferencing an image. Intel NUC consumes 107 W, which is averagely comparable with a server PC, while AGX Orin, Orin Nano, and Raspberry Pi consume less than 20 W of power. The outcomes of our evaluations offer valuable insights for selecting appropriate devices for specific scenarios. These detailed observations on the strengths and limitations of different edge devices can guide future research and advancements in Edge AI technology.