Low-Level Matters: An Efficient Hybrid Architecture for Robust Multi-Frame Infrared Small Target Detection

Multi-frame infrared small target detection (IRSTD) plays a crucial role in low-altitude and maritime surveillance. The hybrid architecture combining CNNs and Transformers shows great promise for enhancing multi-frame IRSTD performance. In the paper, we propose LVNet, a simple yet powerful hybrid architecture that emphasizes low-level feature learning in hybrid frameworks for multi-frame IRSTD. Our key insight is that the standard linear patch embeddings in Vision Transformers are insufficient for capturing the scale-sensitive local features critical to infrared small targets. To overcome this limitation, we design a multi-scale CNN frontend that explicitly models local features by leveraging the local spatial bias of convolution. Additionally, we introduce a U-shaped video Transformer for multi-frame spatiotemporal context modeling, effectively capturing the motion characteristics of targets. To the best of our knowledge, we are the first to introduce the concept of “low-level matters” within the CNN-Transformer hybrid architecture for multi-frame IRSTD. Despite its simplicity, with only 1.77 M parameters and 17.88 G FLOPs, LVNet achieves state-of-the-art (SOTA) performance at a speed of 208 FPS. Experiments on multiple public datasets, comparing LVNet with 18 SOTA methods, demonstrate its superior performance and efficiency. Our code and pre-trained models are accessible at https://github.com/ZhihuaShen/LVNet.