Scalable Fashion Product Retrieval with Multi-Task Fine-Tuned Vision-Language Models and Real-Time Distributed Architecture

This paper presents a novel approach to fashion product retrieval using multi-task fine-tuned vision-language models. We explore and enhance two state-of-the-art models, CLIP and BEiT-3, for efficient retrieval of fashion products based on both text descriptions and image queries. Our contributions include: (1) a multi-task fine-tuning method that maintains contrastive capabilities while adding specialized fashion classification tasks, (2) a distributed system architecture for real-time product retrieval that combines modern technologies including FAISS, Apache Kafka, and Apache Spark, and (3) comprehensive bench-marking that identifies the trade-offs between model accuracy and computational efficiency. Experimental results show significant improvements in retrieval performance, with fine-tuned models achieving up to 6.3% increase in Mean Average Precision compared to baseline models. Our system demonstrates practical real-time performance while handling large product catalogs, making it suitable for production e-commerce environments.