Creating Fair and Accurate AI: The Framework goes beyond that

Two of the most difficult qualities to balance in the world of machine learning are fairness and accuracy. Algorithms that optimize accuracy may inadvertently perpetuate bias against certain groups, while those that prioritize fairness may compromise accuracy by misclassifying some data points.

With this challenge in mind, a team from CSAIL has taken the lead in developing a framework that enables a more nuanced approach to balancing these qualities.

Rather than forcing a binary decision in classifying all data points as “good” or “bad,” their framework uses a rejection option classification (ROC) algorithm that identifies a third category of “rejected samples,” allowing them to identify cases where The model may be less certain or the predictions may be unfair.

By rejecting these cases, the framework can reduce the likelihood of unfair outcomes for certain groups (e.g., by age or gender) without significantly sacrificing overall accuracy.

“Rejected” cases can also be analyzed further to understand potential sources of bias in the data or model. This information can in turn be used to improve the model and data collection process to mitigate bias in the future.

The development of the framework was led by MIT researcher and former professor Amar Gupta, along with affiliate researcher Rashmi Nagpal and College of Engineering student Areeba Khan. Existing systems often focus only on improving “group fairness,” Gupta says, ensuring that protected groups do not discriminate. In contrast, their framework integrates both group justice and individual justice, which involves treating similar individual users similarly.

For example, suppose a machine learning model is used to predict the probability of loan approval for individuals applying for a mortgage. “Group fairness” requires that the model predict loan approval at similar rates for both males and females, ensuring fair treatment between the sexes. In contrast, “individual fairness” means that the model will provide similar predictions for individuals with similar qualifications, regardless of their age, gender, or race.

The team’s experimental analysis comparing the ROC-based framework with similar systems demonstrated its ability to achieve high accuracy and fairness. On a dataset of German credit scores, it achieved over 94% accuracy, meaning the model should make similar predictions for individuals with similar qualifications and circumstances, regardless of sensitive characteristics such as age, gender or ethnicity.

Gupta says that most existing studies conducted in this area have involved public datasets, but the team wanted to explore more private datasets to improve the applicability of debiasing algorithms widely used across many different industry sectors.

“These problems of fairness and equity are not limited to one organization or industry, nor to one isolated worker,” Gupta says. “A tool like ROC can really be used anywhere you need to make discerning judgments about the data you have, from finance to healthcare.”

The team presented their framework as part of paper Published in a special issue of Machine learning and knowledge extraction. A The second related paper (“Improving Fairness and Accuracy: A Pareto Optimal Approach to Decision Making”) was also published earlier this year in the journal Artificial intelligence and ethics magazine.

In the second paper, the researchers worked closely with colleagues at Ernst and Young and other CSAIL affiliates to investigate a decision-making approach based on the economic concept of Pareto optimality. This approach seeks to achieve a state of resource allocation where one aspect of the solution set (i.e. accuracy) cannot be improved without harming another aspect (fairness).

Specifically, the researchers have developed an extension of a framework called Minimax Pareto Fairness (MMPF), which uses a multi-objective loss function that, again, combines elements of group and individual fairness to achieve a Pareto advantage.

The team tested their framework on several open source datasets, including the Adult Census Income, COMPAS, and German Credit datasets, and showed a significant reduction in the accuracy-fairness trade-off across several sensitive features.

The new framework focuses on balancing fairness and performance using two fairness metrics, with plans to explore more in the future. The researchers aim to combine training methods with pre- or post-treatment strategies to enhance results. Next steps include adjusting fairness trade-offs using differential weights, and improving the Pareto reweighting process to assign individual weights to data points for better optimization, Nagpal says.