Human Perceptual Quality Driven Underwater Image Enhancement Framework

Underwater images suffer from severe color casts, low contrast, and blurriness, which greatly degrade the visibility and color fidelity of underwater images. Recently, numerous underwater image enhancement (UIE) algorithms have been proposed. Existing synthetic datasets-based deep learning methods employ synthetic datasets to train UIE models. However, there is a gap between synthetic datasets and real underwater images, leading to a poor generalization of synthetic datasets-based UIE methods. Besides, existing real datasets-based deep learning methods largely focus on minimizing the mean squared reconstruction error between UIE results and corresponding ground truth on the real datasets, but do not take human visual perception into account. Thus, although they achieve a high peak signal-to-noise ratio (PSNR) between UIE results and corresponding ground truth obtained by user study on the real datasets, they often achieve unsatisfactory perceptual quality. To address these problems, we propose a human perceptual quality-driven underwater image enhancement framework (HPQ-UIEF) to achieve better results in human perceptual quality and maintain satisfactory PSNR, which is trained on a real underwater enhancement quality assessment database (UEQAB). Specifically, an underwater image quality assessment network (UIQAN) for UIE images is first proposed to assist the UIE task, in which a novel depth map prior spatial attention block (DPPAB) is embedded into UIQAN. The DPPAB can adaptively recalibrate the quality-aware feature maps and model human visual attention in a data-driven manner. Then, the UIE model is proposed, in which the UIQAN is introduced as the loss function to optimize our UIE model in the direction of perceptual metrics. Moreover, since the confidence map (CM) acquired by UIQAN can effectively reflect the sensitivity of human perceptual of the local area in a UIE image, the confidence map is introduced to our UIEF to help our UIEF to perceive the perceptually important regions. Thus, the confidence map is down-sampled and then concatenated into the decoder module of the UIE model, which can further improve the perceptual quality of the UIE results. Extensive experimental results show that the proposed HPQ-UIEF outperforms state-of-the-art (SOTA) UIE methods qualitatively and quantitatively.