MTMLNet: Multi-Task Mutual Learning Network for Infrared Small Target Detection and Segmentation

Infrared small target detection has been extensively studied due to its wide range of applications. Most studies treat infrared small target detection as an independent task, either as a detection-based or a segmentation-based, failing to fully leverage the supervisory information from different annotation forms. To address this issue, we propose a multi-task mutual learning network (MTMLNet) specifically designed for infrared small targets, aiming to enhance both detection and segmentation performance by effectively utilizing various forms of supervisory information. Specifically, we design a multi-stage feature aggregation (MFA) module capable of capturing features with varying gradients and receptive fields simultaneously. Additionally, a hybrid pooling down-sampling (HPDown) module is proposed to mitigate information loss during the down-sampling process of infrared small targets. Finally, the hierarchical feature fusion (HFF) module is designed to adaptively select and fuse features from different semantic layers, learning the optimal way to fuse features across semantic layers. The results on IRSTD-1k and SIRST-V2 datasets show that our proposed MTMLNet achieves state-of-the-art (SOTA) performance in both detection-based and segmentation-based methods. The codes are available at https://github.com/YangBo0411/MTMLNet