Learning Fashion Compatibility with Color Distortion Prediction

Existing supervised fashion compatibility prediction (SFCP) methods use pre-trained convolutional neural network (CNN) backbones for extracting image representations. However, since CNN backbones are pre-trained for image classification, a task vastly different from fashion compatibility prediction, a feature gap arises that negatively impacts the subsequent learning process. To solve this problem, this paper proposes a color distortion prediction task to prompt the CNN backbone to focus on low-level features. Specifically, we first propose to distort the image by randomly adjusting the image color balance, contrast, sharpness, and brightness. Subsequently, the distorted images are processed through the CNN backbone to discern a probability distribution over all possible distortions. The classification loss between the learned probability distribution and the actual labels is calculated. We experimentally verified the effectiveness of the proposed method in enhancing the prediction accuracy of the SFCP methods. In addition, our approach outperforms state-of-the-art pretext tasks within the domain of fashion compatibility prediction.