Multi-weather restoration: An efficient prompt-guided convolution architecture

Addressing degraded weather conditions plays a vital role in practical applications. Many existing restoration approaches are limited to specific weather types, which limits their applicability to different weather scenarios. Advanced technologies, encompassing Transformer and diffusion model, have been harnessed to confront this challenge. However, these methods often heighten network complexity and prolong inference duration. To this end, we present MW-ConvNet, a U-shaped convolution-based network for multi-weather restoration. Specifically, the MW-Enc block and MW-Dec block are introduced to achieve simple yet strong feature extraction, which rely entirely on traditional 2D convolution. To improve adaptability to multiple weather conditions, a prompt generation module is designed to generate a representative weather prompt at the encoder’s terminus. Drawing inspiration from style transfer, the weather prompt is used to guide the decoder learning through a progressive restoration procedure. For future high-fidelity restoration, we introduce frequency separation through wavelet pooling blocks in encoder phase and corresponding up-sampling blocks in decoder phase. The segregated treatment of low-frequency and high-frequency features curbs the loss of textural information during network computation. It also future improves the quality and accuracy of generated weather prompt. Extensive experiments demonstrate that the proposed MW-ConvNet obtains superior performance compared to state-of-the-art methods across both weather-specific and real-world restoration tasks. Significantly, our method achieves an impressive inference speed of 0.12 seconds per 256x256 image, outpacing transformer-based and diffusion-based models.