Machine Learning Fairness is Computationally Difficult and Algorithmically Unsatisfactorily Solved

The main purpose of the paper is to analyze the computational difficulties of selecting the suitable classification algorithms that satisfy specific ethical criteria, when real data is used in training. Employing an imbalanced credit decision dataset largely used for credit scoring and applying a set of algorithms and several fairness criteria, we show that many typical classification algorithms do not satisfy in a reasonable manner more than one fairness criterion when considering more than one protected attribute. This adds a layer of difficulty to the ones represented by the need of large databases and data- and computationally-intensive decision-making systems as used in domains such as credit scoring and hiring. A novel analysis of this study is directly relating ML/AI fairness criteria and computational complexity. We reframe the problem of complexity by connecting it to the search of an ethically acceptable solution instead of just an accurate solution. The results suggest the continued need for human input in fairness decisions, especially when deciding tradeoffs between fairness criteria.