RGB-Based Gait Recognition With Disentangled Gait Feature Swapping

Gait recognition enables the non-contact identification of individuals from a distance based on their walking patterns and body shapes. For vision-based gait recognition, covariates (e.g., clothing, baggage and background) can negatively impact identification. As a result, many existing studies extract gait features from silhouettes or skeletal information obtained through preprocessing, rather than directly from RGB image sequences. In contrast to preprocessing which relies on the fitting accuracy of models trained on different tasks, disentangled representation learning (DRL) is drawing attention as a method for directly extracting gait features from RGB image sequences. However, DRL learns to extract features of the target attribute from the differences among multiple inputs with various attributes, which means its separation performance depends on the variation and amount of the training data. In this study, aiming to enhance the variation and quantity of each subject’s videos, we propose a novel data augmentation pipeline by feature swapping for RGB-based gait recognition. To expand the variety of training data, features of posture and covariates separated through DRL are paired with features extracted from different individuals, which enables the generation of images of subjects with new attributes. Dynamic gait features are extracted through temporal modeling from pose features of each frame, not only from real images but also from generated ones. The experiments demonstrate that the proposed pipeline increases both the quality of generated images and the identification accuracy. The proposed method also outperforms the RGB-based state-of-the-art method in most settings.