Detecting Fashion Apparels and their Landmarks

Apparel landmarks are the functional key-points on the apparels that can be used for a more discriminative visual analysis of the apparel images. Such a framework can facilitate apparel alignment in displaying apparel images on the websites for recommendation systems, apparel image retrieval, apparel style transfer, trend analysis or help build a system to ensure dress code in a particular environment. However, challenges such as background clutter, human poses, scales apparel variation and lighting can render such a task difficult. We present a conceptually simple, flexible, and general framework for apparels' landmark detection that can also be simultaneously used for apparel detection. In addition to the position of the landmarks in the apparels, we also classify the landmarks as visible or occluded in the same framework. We perform all these tasks in parallel using multi-task learning. Our proposed convolutional neural network is end-to-end differentiable and simple to train, since all these tasks are performed on the same architecture without any additional parameters to learn. The fashion landmark detection task is similar to joint localization and detection problems like human pose estimation, hence our approach extends stacked hourglass architecture, originally proposed to solve human pose estimation. Over the past few years, many modifications have been proposed to improve this architecture. We also compare the performances of some of these different variations of stacked hourglass architectures. These architectures leverage both global and local features captured by the deep convolutional neural networks to better localize the apparel in the image as well as the landmarks in those apparels.