Cross-Modal Learning Based Flexible Bimodal Biometric Authentication With Template Protection

Face and voice are two of the most popular traits used for authentication tasks in daily life, as they can be easily captured using low-cost visual and audio sensors on smartphones, laptops, tablets, etc. Many bimodal biometric authentication schemes based on these two traits have been presented to provide higher accuracy than unimodal systems. However, these schemes are inflexibility due to the requirement of submitting two traits simultaneously, and they lack template protection, which may lead to biometric data leakage. We present a cross-modal learning based bimodal biometric authentication scheme, which improves the flexibility of existing schemes while ensuring the biometric template security. We integrate cross-modal learning into the feature extraction to obtain a bimodal biometric shared representation given input face images and voice clips. In order to enhance biometric template security without sacrificing authentication accuracy, a residual network and polar codes based template protection method is proposed, which can eliminate the noise in shared representations due to intra-user variations and generate protected templates. We have evaluated the efficacy of the bimodal biometric scheme using a real video dataset containing face images and voice clips. Experimental results demonstrate that our scheme can achieve flexible authentication with high accuracy no matter the probe input is a face image, a voice clip or a combination of them. Furthermore, the security analysis demonstrates that our scheme provides irreversibility, unlinkability and revocability of protected templates.