Feature Affinity based Clustering for Test-Time Adaptation for Image Quality Assessment

Recently, Test-Time Adaptation (TTA) algorithms have gained traction in image/video quality assessment (IQA/VQA). These methods use virtual losses, like group contrastive and rank loss, as auxiliary tasks to adapt batch normalization parameters, helping models generalize better to distribution shifts between training and testing datasets. Group contrastive loss clusters images into low- and high-quality groups based on predicted quality scores, maximizing feature space distance between them. However, its effectiveness relies on the base model’s ability to accurately predict quality scores, which can be compromised when distribution shifts occur, leading to suboptimal adaptation and degraded performance. We propose a novel clustering approach based on the assumption that high-quality images contain richer high-level information, which is extracted using a pre-trained VGG-16 model. Images are clustered by comparing the VGG-16 features of the highest quality image in a batch with the others, enabling effective grouping based on feature affinities. These accurate clusters enhance the computation of contrastive loss, improving the adaptation of batch normalization layers. Additionally, we introduce an adaptive rank loss to reduce the impact of rank loss when the base model can distinguish images with varying distortion levels. Experimental results across multiple image quality assessment datasets, including LIVE, CID-2013, KONIQ-10K, and SPAQ, as well as algorithms like MetaIQA, HyperIQA, TReS, and MUSIQ, show that the proposed method consistently performs better than the existing Test-Time Adaptation (TTA) approach.