Moritz Hardt, Eric Price, Nathan Srebro
2016 · arXiv
We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. In line with other studies, our notion is oblivious: it depends only on the joint statistics of the predictor, the target and the protected attribute, but not on interpretation of individualfeatures. We study the inherent limits of defining and identifying biases based on such oblivious measures, outlining what can and cannot be inferred from different oblivious tests. We illustrate our notion using a case study of FICO credit scores.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
Differences in lending rates between groups due to less data or confounding features is the motivating example in the oft-cited 'Equality of Opportunity in Supervised Learning'. Highly recommend it: https://arxiv.org/abs/1610.02413
Difficult to choose one this year (plenty of things happened + plenty of time to read things due to lock down). "Equality of Opportunity in Supervised Learning" (https://arxiv.org/abs/1610.02413) It explain the basic concept about fairness in ML. Very practical exemple in my domain knowledge that show the trade-off between fairness of an algo and overall performance (money). Really make you see what may go wrong with bias in ML. It shows, in my opinion, why we will have to regulate ML as corporation aren't really incentivized to deal with fairness. It also shows that there is different notions of fairness. So there will always be something that feel unfair and also doing something can always be interpreted as positive discrimination.
> Some people expect that fighting algorithmic racism is going to come with some sort of trade-off. Um, that's because we know it comes with trade-offs once you have the most optimal algorithm. See for instance https://arxiv.org/pdf/1610.02413.pdf. If your best performing algorithm is "racist" (for some definition of racist") you are mathematically forced to make tradeoffs if you want to eliminate that "racism". Of course, defining "racism" itself gets extremely tricky because many definitions of racism are mutually contradictory (https://arxiv.org/pdf/1609.05807.pdf).
> The goal should be equality of opportunity and evaluation rather than equality of outcome. In general, that will still require taking race into account. https://arxiv.org/abs/1610.02413 "Equality of Opportunity in Supervised Learning" discusses various fairness measures in the context of deciding loan applications based on credit scores. The issue is that you only know whether someone was qualified once they pay the loan back, and credit scores are not a perfect predictor of that, which on top of that is differently imperfect for different races. If you want equal opportunity (defined as "the probability of a qualified application being accepted is equal independent of race"), you need different thresholds to compensate for the different prediction error rates.
Google has actually released a paper on codifying what discrimination looks like for lending (https://arxiv.org/abs/1610.02413 + blog post: https://research.googleblog.com/2016/10/equality-of-opportun...). It can be trivially extended to discuss hiring. Basically a fair hiring process as defined by the above would be: P(hire | features) = P(hire | race=1, features) = P(hire | race=2, features) This may sound a bit silly or unnecessary, but unless this formula is specifically enforced, if a race were correlated with expected profit of a hire, a hiring algorithm might accidentally infer race from the other features. You can see the paper for an extended discussion. With…
that background out of the way, let's consider hiring based on quotas. Since hiring takes place continuously across time, in order to maintain a quota across time we would need to change P(hire | race=1) based on the current composition of the company. Therefore P(hire | race=1) != P(hire | race=2) at least some of the time. Therefore it's discriminatory according to the above definition. Of course, you may not like the definition. I would be very interested to hear an alternate formalization of non-discrimination!
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Anders Sandberg, Eric Drexler, Toby Ord
The Fermi paradox is the conflict between an expectation of a high {\em ex ante} probability of intelligent life elsewhere in the universe and the apparently lifeless universe we in fact observe.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.