BAL-Net: Bi-Attention Learning for Single Image De-raining

Within the image-to-image translation framework, single image de-raining can be addressed by training a network with pairs of rainy and clean images, and training the network can be interpreted as learning to distill knowledge from the clean contextual regions to predict the corrupted pixels. To overcome the problem of over or under de-raining with existing models, a bi-attention mechanism is formulated for better rainy image restoration including both raindrop and rain streak removal. Specifically, the contextual attention map (directing where to focus on the background scenes) and rainy attention map (indicating corrupted regions to process) are firstly learned, and are then concatenated with the rainy image as input to the de-raining network for more effective representation learning and rainy image restoration. To better model the non-uniformly distributed raindrops or rain streaks and generate detail-preserving de-rained images, a multi-branch multi-scale module is proposed and incorporated into the dense residual blocks, which are stacked for constructing the bi-attention guided single image de-raining network. Comprehensive evaluation on both synthetic and real rainy datasets demonstrates the superiority of the proposed models over other methods.