STDP-Net: Improved Pedestrian Attribute Recognition Using Swin Transformer and Semantic Self-Attention

An image location requiring focus to recognize a specific pedestrian attribute often depends on the state of the pedestrian within the image. In addition, various pedestrian attributes are closely related to each other. For example, the “Boots” and “ShortSkirt” attributes are related to the “Female” attribute. For these reasons, we propose a novel encoder-decoder network for pedestrian attribute recognition, called Swin Transformer and Decoder for Pedestrian attribute recognition Network (STDP-Net). First, we utilize a Swin Transformer that uses self-attention as the encoder. This allows the proposed method to understand the relative relationship between the spatial regions of the images, unlike conventional convolution-based methods. This enables an accurate recognition of the attributes, even in misaligned pedestrian image inputs. Second, we add a transformer decoder with learnable attribute queries to the encoder to understand the semantic relationships among the attributes. Using the decoder, the proposed method captures such relationships based on the self-attention of the attribute queries. Extensive experimental results demonstrate that the proposed method achieves a state-of-the-art performance on six pedestrian attribute recognition datasets. In addition, misalignment experiments on the PETA, PA100K, and RAP datasets show the superiority of the encoder-decoder structure in comparison with other state-of-the-art methods.