Improving Fashion Attribute Classification Accuracy with Limited Labeled Data Using Transfer Learning

Classification of fashion item attributes, such as neckline types, graphic patterns, sleeve lengths, hem types/length, etc., plays an essential role in the fashion item recommender systems by means of providing precise categorization and recommendations. With recent advances in Artificial Intelligence, many online retailers have adopted deep learning models to detect and classify the attributes of their product images effectively. However, these deep learning models require large and diverse datasets with labels to achieve an acceptable accuracy, which takes time and effort to collect and annotate. Another challenge of deep learning is its limited generalization capability to a new type of sample data. We hypothesize that imbuing a neural classification model with a human’s cognitive capability, such as recognizing complex patterns by simple geometric shapes, can improve fashion attribute classification performance. This paper proposes a transfer learning-based image classification model that exploits synthetic examples of canonical shapes (e.g., circles, triangles, rectangles, etc.) from publicly available datasets and our examples created using simple graphic tools. We use these datasets of geometric shapes as a source domain to pre-train a model and fine-tune it with labeled images to solve the target problem (i.e., fashion attribute classification). Our proposed framework increases the accuracy of the neckline type and graphical pattern classifications of Resnet50 by 40.7% and 19.8%, from 49.4% and 54.8% to 90.1% and 74.6%, respectively.