Hue Are You? Can Skin Depigmentation Affect Face Recognition Performance?

Vitiligo causes localized loss of skin pigmentation, producing visible tone variations that may affect face recognition accuracy. While skin tone is a known covariate, the specific impact of vitiligo has not been studied. This work addresses two key questions: (1) does the presence of vitiligo present a challenge for face recognition systems, and (2) does the severity of vitiligo influence performance? First, we introduce Vitiligo Faces (VF), a new dataset of individuals with vitiligo, and compare matcher performance on VF against CFPW, a benchmark composed of difficult image pairs. Two of the three evaluated matchers perform worse on VF than CFPW, with greater score overlap and higher false non-match rates—indicating that vitiligo presents a unique and substantial challenge. Second, we develop a computational pipeline to quantify severity and find that increasing severity is associated with declining performance, as measured by d-prime (d’) separability. These findings underscore the need to evaluate recognition systems under underrepresented, clinically relevant conditions. Vitiligo introduces real-world variation that current models are not well equipped to handle, highlighting the importance of inclusive training and evaluation.