OSFormer: One-Step Transformer for Infrared Video Small Object Detection

Infrared video small object detection is pivotal in numerous security and surveillance applications. However, existing deep learning-based methods, which typically rely on a two-step paradigm of frame-by-frame detection followed by temporal refinement, struggle to effectively utilize temporal information. This is particularly challenging when detecting small objects against complex backgrounds. To address these issues, we introduce the One-Step Transformer (OSFormer), a novel method that pioneeringly integrates a small-object-friendly transformer with a one-step detection paradigm. Unlike traditional methods, OSFormer processes the video sequence only through a single inference, encoding the sequence into cube format data and tracking object motion trajectories. Additionally, we propose the Varied-Size Patch Attention (VPA) module, which generates patches of varying sizes to capture adaptive attention features, bridging the gap between transformer architectures and small object detection. To further enhance detection accuracy, OSFormer incorporates a Doppler Adaptive Filter, which integrates traditional filtering techniques into an end-to-end neural network to suppress background noise and accentuate small objects. OSFormer outperforms YOLOv8-s on both the AntiUAV dataset (+ $3.1\%~\text {mAP}_{50}$ , - $35.1\%~\text {Params}$ ) and the InfraredUAV dataset (+ $4.0\%~\text {mAP}_{50-95}$ , - $51.0\%~\text {FLOPs}$ ), demonstrating superior efficiency and effectiveness in small object detection. The code is available on https://github.com/q2479036243/OSFormer.