Abstract:
Near-infrared and visual (NIR-VIS) face matching, as the most typical task in Heterogeneous Face Recognition (HFR), has attracted increasing attention in recent years. Ho...Show MoreMetadata
Abstract:
Near-infrared and visual (NIR-VIS) face matching, as the most typical task in Heterogeneous Face Recognition (HFR), has attracted increasing attention in recent years. However, due to the large within-class discrepancies, including domain differences and residual discrepancies (i.e., lighting, expressions, occlusion, blurry, pose, etc), this is still a difficult task. Conventional NIR-VIS FR methods only focus on reducing the modality gap between cross-domain images, while neglecting to eliminate the residual variations. To better solve the above problems, this paper proposes a novel Orthogonal Modality Disentanglement and Representation Alignment (OMDRA) approach, which consists of three key components, including Modality-Invariant (MI) loss, Orthogonal Modality Disentanglement (OMD) and Deep Representation Alignment (DRA). Firstly, the MI loss is designed to learn modality-invariant and identity-discriminative representation, by increasing between-class separability and within-class compactness between NIR and VIS heterogeneous data. Secondly, the high-level Hybrid Facial Feature (HFF) layer of the backbone network is projected into two subspaces: the modality-related and identity-related subspaces. The OMD is designed to decouple modal information via an adversarial process, and we further impose Orthogonal Representation Decorrelation (ORD) to the OMD to decrease the correlation between identity representations and domain representations, as well as enhancing their representation capabilities. Finally, the DRA aims to eliminate the residual variations by performing a high-level representation alignment between non-neutral face and neutral face, which can effectively guides the network to learn discriminative and residual-invariant face representation. The joint scheme enables the disentanglement of modality variations, elimination of residual discrepancies, and the purification of identity information. Extensive experiments on challenging cross-domain databases ...
Published in: IEEE Transactions on Circuits and Systems for Video Technology ( Volume: 32, Issue: 6, June 2022)
Funding Agency:
References is not available for this document.
Contents Select All
1.
C. Peng, X. Gao, N. Wang, and J. Li, “Superpixel-based face sketch–photo synthesis,” IEEE Trans. Circuits Syst. Video Technol., vol. 27, no. 2, pp. 288–299, Feb. 2017.
2.
X. Gao, J. Zhong, J. Li, and C. Tian, “Face sketch synthesis algorithm based on E-HMM and selective ensemble,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 4, pp. 487–496, Apr. 2008.
3.
B. F. Klare and A. K. Jain, “Heterogeneous face recognition using kernel prototype similarities,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1410–1422, Jun. 2013.
4.
Z. Lei and S. Z. Li, “Coupled spectral regression for matching heterogeneous faces,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1123–1128.
5.
D. Yi, Z. Lei, and S. Z. Li, “Shared representation learning for heterogenous face recognition,” in Proc. 11th IEEE Int. Conf. Workshops Autom. Face Gesture Recognit. (FG), May 2015, pp. 1–7.
6.
R. He, X. Wu, Z. N. Sun, and T. N. Tan, “Wasserstein CNN: Learning invariant features for NIR-VIS face recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 7, pp. 1803–1816, Jun. 2019.
7.
R. He, X. Wu, Z. Sun, and T. Tan, “Learning invariant deep representation for NIR-VIS face recognition,” in Proc. AAAI Conf. Artif. Intell., 2017, pp. 2000–2006.
8.
F. J. Xu, D. K. Pal, and M. Savvides, “NIR-VIS heterogeneous face recognition via cross-spectral joint dictionary learning and reconstruction,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, Jun. 2015, pp. 141–150.
9.
W. Hu and H. Hu, “Dual adversarial disentanglement and deep representation decorrelation for NIR-VIS face recognition,” IEEE Trans. Inf. Forensics Security, vol. 16, pp. 70–85, 2021.
10.
J. Lu, V. Liong, and J. Zhou, “Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 8, pp. 1979–1993, Aug. 2018.
11.
M. Zhang, R. Wang, X. Gao, J. Li, and D. Tao, “Dual-transfer face sketch-photo synthesis,” IEEE Trans. Image Process., vol. 28, no. 2, pp. 642–657, Feb. 2019.
12.
X. Huang, Z. Lei, M. Fan, X. Wang, and S. Z. Li, “Regularized discriminative spectral regression method for heterogeneous face matching,” IEEE Trans. Image Process., vol. 22, no. 1, pp. 353–362, Jan. 2013.
13.
M. Kan, S. Shan, H. Zhang, S. Lao, and X. Chen, “Multi-view discriminant analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 1, pp. 188–194, Jan. 2012.
14.
X. Li, “DeepSaliency: Multi-task deep neural network model for salient object detection,” IEEE Trans. Image Process., vol. 25, no. 8, pp. 3919–3930, Aug. 2016.
15.
X. Wu, R. He, Z. Sun, and T. Tan, “A light CNN for deep face representation with noisy labels,” 2015, arXiv:1511.02683. [Online]. Available: http://arxiv.org/abs/1511.02683
16.
X. Wu, R. He, Z. Sun, and T. Tan, “A light CNN for deep face representation with noisy labels,” IEEE Trans. Inf. Forensics Security, vol. 13, no. 11, pp. 2884–2896, Nov. 2018.
17.
O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in Proc. Brit. Mach. Vis., vol. 1, no. 3, 2015, pp. 6–17.
18.
S. Saxena and J. Verbeek, “Heterogeneous face recognition with CNNs,” in Proc. Eur. Conf. Comput. Vis. Workshops, Oct. 2016, pp. 483–491.
19.
X. Liu, L. Song, X. Wu, and T. Tan, “Transferring deep representation for NIR-VIS heterogeneous face recognition,” in Proc. Int. Conf. Biometrics (ICB), Jun. 2016, pp. 1–8.
20.
C. Peng, N. Wang, J. Li, and X. Gao, “Re-ranking high-dimensional deep local representation for NIR-VIS face recognition,” IEEE Trans. Image Process., vol. 28, no. 9, pp. 4553–4565, Sep. 2019.
21.
T. de Freitas Pereira, A. Anjos, and S. Marcel, “Heterogeneous face recognition using domain specific units,” IEEE Trans. Inf. Forensics Security, vol. 14, no. 7, pp. 1803–1816, Jul. 2018.
22.
X. Wu, L. X. Song, R. He, and T. N. Tan, “Coupled deep learning for heterogeneous face recognition,” in Proc. AAAI Conf. Artif. Intell., 2018, pp. 1679–1686.
23.
Z. Deng, X. Peng, Z. Li, and Y. Qiao, “Mutual component convolutional neural networks for heterogeneous face recognition,” IEEE Trans. Image Process., vol. 28, no. 6, pp. 3102–3114, Jun. 2019.
24.
Z. Y. Deng, X. J. Peng, and Y. Qiao, “Residual compensation networks for heterogeneous face recognition,” in Proc. AAAI Conf. Artif. Intell., 2019, pp. 8239–8246.
25.
W. Hu and H. Hu, “Adversarial disentanglement spectrum variations and cross-modality attention networks for NIR-VIS face recognition,” IEEE Trans. Multimedia, vol. 23, pp. 145–160, 2021.
26.
W. Hu, W. Yan, and H. Hua, “Dual face alignment learning network for NIR-VIS face recognition,” IEEE Trans. Circuits Syst. Video Technol., early access, May 18, 2021, doi: 10.1109/TCSVT.2021.3081514.
27.
S. Z. Li, D. Yi, Z. Lei, and S. Liao, “The CASIA NIR-VIS 2.0 face database,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, Jun. 2013, pp. 348–353.
28.
J. Chen, D. Yi, J. Yang, G. Zhao, S. Z. Li, and M. Pietikainen, “Learning mappings for face synthesis from near infrared to visual light images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 156–163.
29.
D. Huang, J. Sun, and Y. H. Wang, “The BUAA-VisNir face database instructions,” Beihang Univ., Beijing, China, Tech. Rep. IRIP-TR-12-FR-001, 2012.
30.
T. Ahonen, A. Hadid, and M. Pietikäinen, “Face description with local binary patterns: Application to face recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 12, pp. 2037–2041, Dec. 2006.
