From Fusion to Adaptation: Investigation on Enhancing Multimodal Biometric Authentication Systems

Multimodal biometric authentication system has been proposed to mitigate issues inherent in systems that rely on a single biometric trait. Studies on multimodal biometric authentication systems have compared different fusion methods to find an optimal fusion level to combine different biometric traits; however the integration of facial and keystroke biometric samples have not been fully investigated. In addition, some recent studies on biometric authentication systems have shown the importance of adaption where systems can mitigate issues of performance degradation over time in dynamic environments. The application of heterogeneous multimodal contexts remains under explored - this study, therefore first investigates the fusion of facial and keystroke biometric traits by comparing feature and score level fusion methods utilising three virtual datasets: Face with CMU, Face with GREYC and Face with Mobile. The findings demonstrate that score-level fusion performs better in terms of accuracy compared to feature-level fusion. Leveraging this result, the study further applies an adaptation mechanism together with score-level fusion. The study updates thresholds and templates by comparing different strategies with the proposal of a hybrid strategy - Dual Path Adaptation and Positive Gallery Protection Adaptation. Instead of relying only on the threshold or using genuine biometric samples alone to determine adaptation, the hybrid strategy considers both imposter and genuine biometric references. Using information from both sources, the hybrid strategy strengthens adaptation decision making to improve recognition accuracy. The results demonstrate the effectiveness of different adaptation methods across three virtual datasets. Dual Path Adaptation achieved the highest performance with recognition accuracy scores of 0.7136 (Face with CMU), 0.6925 (Face with GREYC), and 0.7862 (Face with Mobile), where scores represent the proportion of correctly identified instances (e.g., 0.7136=71.36%). Positive Gallery Protection Adaptation followed with 0.6852, 0.7281, and 0.7422, respectively. Adaptive Dual Thresholds yielded 0.7050, 0.6506, and 0.7289, while the baseline without adaptation scored 0.7123, 0.7157, and 0.7499. These findings highlight the importance of selecting an appropriate adaptation strategy, as an unsuitable approach can degrade performance and compromise security. Therefore, it is crucial to identify a well-balanced adaptation method that enhances robustness without sacrificing security.