Haformer: Heterogeneous Aggregation Transformer for Single Image Deraining

Transformer-based methods have demonstrated impressive achievements in the realm of single image deraining (SID). Nonetheless, these advancements still confront a significant challenge: the employed self-attention mechanisms facilitate homogeneous feature aggregation, consequently restricting the capacity for holistic information collection across both spatial and channel dimensions. To surmount the above limitation, we introduce an innovative approach for SID termed Heterogeneous Aggregation Transformer (HAformer). The core of our proposed HAformer lies the Heterogeneous Aggregation Attention (HAA) mechanism. Diverging from conventional Transformers employed in SID, which employ unidimensional self-attention for homogeneous modeling, our proposed HAA orchestrates heterogeneous aggregation through a dual-branch architecture. More specifically, these two branches are tailored to achieve simultaneous information extraction across both channel and spatial dimensions. Extensive experiments on several datasets demonstrate that our HAformer yields superior visual results in rain removal and effectively outperforms state-of-the-art methods. Our code is available at https://github.com/stephensun11/HAformer.git.