Object Detection in Drone Video with Temporal Fusion Network

Images acquired by Unmanned Aerial Vehicles (UAV) suffer from serious deterioration. A typical solution for UAV based image object detection is to enhance target appearance in current frame by aggregating the features in neighboring frames. Simple feature aggregation methods will introduce interference of background into targets. To deal with the problem, a simple but effective Temporal Fusion Network (TFN) is proposed. The key idea is to use separable 3D convolutions and deformable convolutions to align features in different frames. TFN works as an add-on module to bring existing static object detectors to high performance video object detectors with negligible extra computational cost. Exhaustive experiments are carried both on the challenging VisDrone2019-VID dataset and the large scale ImageNet-VID dataset based on different image object detectors, i.e., the anchor-based YOLOv7 detector and the anchor-free YOLOX detector. Particularly, our network based on YOLOv7, termed TFN-YOLOv7, not only achieves a 1.84% increase in mean Average Precision (mAP), but also reaches a running speed of 23 frames per second (fps).