Attention to Task-Aligned Object Detection for End–Edge–Cloud Video Surveillance

Edge computing extends computing power down to the network edge near the sensors and provides promising solutions to address high bandwidth requirements and latency-sensitive problems in smart video surveillance. Deep learning-based object detection has made significant progress in accuracy and is playing an important role in video surveillance. However, the computation and storage resources are still far below the advances reached by cloud servers, making it difficult to migrate traditional object detection models to the resource-constrained edge. Thus, in this article, we propose a new lightweight object detection model, called attention task-aligned YOLO (AT-YOLO) to detect key targets in the video stream. The efficient single compressed dense block (SCDB) is introduced to build the backbone network of AT-YOLO, which improves the computational efficiency of the network. Besides, the attention guide task-aligned head (AGT-head), which enhances the classification and the localization features, and encourages the interaction and the alignment of both, is proposed to improve the detection accuracy. Considering the redundancy in video streams and the improvement of detection efficiency, a convolutional neural network (CNN)-based keyframe algorithm is considered as it has only 0.32 M parameters and can accurately and quickly filter out keyframes. We deployed the CNN-based keyframe algorithm and AT-YOLO on the edge devices of the system to implement an intelligent surveillance system based on edge computing. Experiments and evaluations on COCO demonstrate that AT-YOLO achieves 29.9% mAP at 3.47M parameters. Moreover, in experiments with real devices, our approach can reduce the bandwidth requirement to 467.29 kb/s. And it achieves a processing speed of 13.69 FPS on the Raspberry Pi 4B, which is a single-board computer. With the more powerful device selected, the better real-time performance it can get.