Estimating Group Fairness Using Pairwise Similarity

Ensuring group fairness in machine learning models requires measuring model fairness performance under various metrics, typically defined under explicit $g$ roup membership information. In FAccT'23, Liu, Do, Usunier, and Nickel proposed a “group-free” fairness measurement based on homophily in social networks. They derived the measurement based on generalized entropy indices, typically used to measure inequality in economics, using pairwise similarity estimation from social connections. This measurement has one major drawback, i.e., the unfairness quantity is group-weighted. While this gives a number that represents the overall picture, it might not give enough protection for minority groups. We generalize their approach based on pairwise similarity and define relaxed group fairness measurements based on many standard group fairness metrics. We provide theoretical justifications by proving that these measurements are indeed relaxations of standard ones, and demonstrate the practicality of our approach with experiments.