An End-to-End Deep Learning System for Automated Fashion Tagging: Segmentation, Classification, and Hierarchical Labeling

This paper presents a comprehensive end-to-end system designed for the automated tagging of fashion products using deep learning and image processing techniques. The system initiates by segmenting fashion items in images to define their boundaries, achieving a mean Intersection-over-Union (IoU) score of $0.79\pm 0.24$ with Detectron2, outperforming alternative models such as U-Net, which scored an IoU of $0.62\pm 0.36$ . Following segmentation, the system classifies these items into primary categories, reaching a top-level classification accuracy of 96% with the InceptionV3 model, and determines their dominant color with an RGB-based color-matching algorithm. The hierarchical structure enables multi-tier labeling, refining attributes like skirt length, collar type, and sleeve length. Once the primary category and color are identified, the product is further categorized using a taxonomy tree that includes additional classifiers, allowing hierarchical tagging with an overall tagging accuracy of 56%. Although this accuracy may seem modest, it is significant given the high number of classes, the hierarchical nature of the label tree, and the challenging presence of cropped or partially visible products in images, all of which naturally increase the error rate. The system is deployed on Google Cloud Run, demonstrating real-world feasibility with an average request latency of 67ms. Its scalable architecture supports high-traffic e-commerce applications. This detailed product labeling process not only improves operational efficiency for e-commerce platforms in the fashion industry but also contributes to improved product discoverability and customer satisfaction, supporting a stronger competitive position in online fashion retail.