IPT-ILR: Image Pyramid Transformer Coupled With Information Loss Regularization for All-in-One Image Restoration

All-in-one image restoration has recently developed to be a new research trend in the low-level computer vision field, aiming to tackle multiple image degradation types simultaneously in a unified model. As a typical multi-task learning, existing approaches focus on modeling either the specificity or commonality among different image restoration tasks. To exploit the unique strengths of both worlds, we propose a method of Image Pyramid Transformer coupled with Information Loss Regularization (IPT-ILR), in which the multi-scale architecture structure can excavate more information for multiple restoration tasks concurrently, while the learning strategy can identify the difference among multiple restoration tasks depending on the degree of information loss in each restoration task. Specifically, it first establishes a new Image Pyramid Transformer Network (IPT-Network) to accommodate multiple image restoration tasks. Given original degraded images, the IPT-Network exploits the image pyramid technique to establish a series of images with different scales, which are then restored by transformer-like auto-encoders. Moreover, the restored image on a low-level scale is referenced to assist restoring the degraded image on a high-level scale. Next, Information Loss Regularization (ILR) is presented to optimize the IPT-Network. ILR calculates the average distance between degraded images and their clean counterparts as the weights, which automatically implement different penalties for different image restoration tasks, thus avoiding the short-cut phenomenon for the easy task while encouraging the hard task. Extensive experiments have been conducted with 6 image restoration tasks in the all-in-one setting. The results show our method performs favorably against numerous state-of-the-art methods across most tasks, including image denoising, image deblurring, image dehazing, image deraining, image desnowing, as well as low-light enhancement.