Adapt-FuseNet: Context-aware Multimodal Adaptive Fusion of Face and Gait Features using Attention Techniques for Human Identification

Biometrics plays a significant role in vision-based surveillance applications. Soft biometrics such as gait is widely used with face in surveillance tasks like person recognition and re-identification. Nevertheless, in practical scenarios, classical fusion techniques respond poorly to the changes in individual users, external environment and varying contexts such as viewpoints. To this end, we propose a novel context-aware adaptive multi-biometric fusion strategy viz., ‘Adapt-FuseNet’ for the dynamic incorporation of gait and face biometric cues leveraging attention techniques. In particular, we investigate the impact of attention models such as parallel co-attention & keyless attention, along with various fusion strategies such as naïve fusion & adaptive fusion for human identification. Extensive experiments are carried out on two publically available large gait datasets i.e. CASIA-A and CASIA-B. Results show the superior performance of our proposed context-aware adaptive fusion model compared with the state-of-the-art models.