The increasing influence of machine learning algorithms and artificial intelligence on the high-impact domains of decision making has led to an increasing concern for the ethical and legal challenges posed by the sensitive data-driven systems. Machine learning can identify the statistical patterns in the historically collected big data generated by a huge number of instances that might be affected by human and structural biases. ML algorithms have the potential to amplify these inequities.
Lately, there have been several attempts to reduce bias in artificial intelligence in order to maintain fairness in machine learning projects. These methods fall under three categories of pre-processing, in-processing and post-processing techniques. There are at least 21 notations of fairness in the recent literature, which not only provide different measurement methods of fairness but also lead to completely different concepts. It is worth mentioning that, it is impossible to satisfy all of the definitions of fairness at the same time and some of them are incompatible with each other. As a result, it is important to choose a fairness definition that need to be satisfied according to the context that we are working on.
The current study investigates some of the most common definitions and metrics for fairness introduced by researchers to compare three of the proposed de-biasing techniques regarding their effects on the performance and fairness measures through empirical experiments on four different datasets. The de-biasing methods include the “Reweighing Algorithm”, “Adversarial De-biasing Method”, the “Reject Option Classification Method” performed on the classification tasks of “Survival of patients with heart failure”(Heart Failure Dataset), “Prediction of hospital readmission among diabetes patients” (Diabetes Dataset), “Credit classification of bank account holders” (German Credit Dataset), and “The COVID19 related anxiety level classification of Canadians” (CAMH Dataset).
This project is performed using the AI Fairness 360 toolkit.