Multi-Modal Gait Recognition with Unidirectional Cross-modal Alignment

Gait recognition represents a pivotal challenge in visual signal comprehension, encompassing multi-modal spatial-temporal information, and exhibiting considerable complexity. The prevailing gait recognition methods are categorized into appearance-based and model-based, which commonly use silhouette and skeleton as input respectively. Certain recent studies have endeavored to integrate both of these modalities, achieving a certain degree of success in doing so. However, the aforementioned methods either rely solely on a modal of information or lack a profound consideration of the intricate interplay among multiple modal. Consequently, the potential inherent in gait’s spatial-temporal information remains untapped. To address this issue, this study strives to enhance the efficiency of data utilization through the integration of multi-modal information within a multi-modal fusion network. Furthermore, we propose a scheme aimed at enhancing the consistency between distinct modal features. Experiments on the widely used gait dataset CASIA-B have shown that our model has significantly improved under complex gait conditions, with overall performance reaching the most advanced level.