Skip to content
The methodOpen data
Back to the collection

European Union regulations on algorithmic decision-making and a "right to explanation"

Bryce Goodman, Seth Flaxman

2016 · arXiv

AlgorithmsPsychologyMachine learning
Read the paper
HN score
0
Mentions
8
Readers
8
Months
6
What’s in a score?

From the abstract

We summarize the potential impact that the European Union's new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which "significantly affect" users. The law will also effectively create a "right to explanation," whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for computer scientists to take the lead in designing algorithms and evaluation frameworks which avoid discrimination and enable explanation.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

8 selected excerpts · 8 total mentions

denzil_correa
Neutral mention · rule
Along with the GDPR, the EU also bought "Right to Explanation" for algorithmic decision making. Basically, a user has the right to ask for an explanation on an algorithmic decision. This regulation has been mostly buried in the deluge of GDPR content but it's an important shift in decision making. There's a nice paper which outlines the implications and HN too had a discussion on it couple of years ago [0, 1]. [0] https://arxiv.org/abs/1606.08813 [1] https://news.ycombinator.com/item?id=12048223
Original comment How Do Insurance Companies Know My Health History? (2017)
dspoka
Neutral mention · rule
Europe has also passed “right to explanation” to help with this issue which is great. unfortunately it seems they came up with it in isolation from researchers on this topic and went with the don't record it philosophy. here is a good resource on that: https://arxiv.org/pdf/1606.08813.pdf European Union regulations on algorithmic decision-making and a “right to explanation”
Original comment Algorithmic decision making and the cost of fairness (2017)
blueblob
Neutral mention · rule
This is an area that fades in and out of focus with such venues as the Workshop on Human Interpretability in Machine Learning (WHI) [1]. It's becoming increasingly important when it comes to auditability and understanding of what is actually learned by algorithm. Avoiding classifiers from learning to discriminate based on age, race, etc [2] or in domains where it's important to know what the algorithm is doing such as medicine. Work in understanding DL is not really interpretable in any domain, typically they train another (simpler, less accurate) model and use that to explain what the model is doing or use perturbation analysis to try to tease out what it is learning. If…
Read the full comment
all you care about is getting the right answer and not why you get that answer maybe it doesn't matter. I wouldn't say I'm hating on DL nor that I hate on random forests, or ensembles, etc., but when you have very little data fitting an uninterpretable, high dimensional model might not be the right answer, in my opinion, see [3]. [1] https://arxiv.org/html/1607.02531v2 [2] https://arxiv.org/abs/1606.08813 [3] https://arxiv.org/abs/1601.04650
Original comment You can probably use deep learning even if you don't have a lot of data
agentcoops
Neutral mention · rule
Lives of members of social groups can be different for historical reasons. Because of this, the best selection of features (as in, how we select and encode relevant aspects of a dataset for a particular problem) that one would need to use, as well as the correlations between them, may be different between different groups. The question is not whether there is fundamentally a difference between, in this case, racial groups vis-a-vis paying back loans (i.e. that the only feature required in your model would be "is member of this group"), but what are the traits of possibilities of life that have for whatever reason ended up leaving a quantifiable trace in our databases and…
Read the full comment
their distribution within that group. One hypothetical example: suppose that there existed a group G that was not able to go to the top n% of universities due to discrimination. Your company uses some rank of university attended as one of the features input to its favorite machine learning algorithm. However, the dataset you trained on excluded group G. Within this group, the best university individuals have been able to attend is X which is by definition not in the top n%. Had the algorithm been trained on this group it would have observed that school X is highly correlated with success in this group, even if not in the original training set used. As is, your ML system assigns a low probability to members of group G. Issues like this will be hard to prevent. While that doesn't mean we shouldn't work hard to make real innovations in ML, I think the legal approach of a "right to explanation" as analyzed in http://arxiv.org/pdf/1606.08813v3.pdf and recently added to European law is regardless a helpful tool to ensure accountability.
Original comment Approaching fairness in machine learning
tlb
Neutral mention · rule
Another recent paper on this topic: http://arxiv.org/pdf/1606.08813v3.pdf. It shows how naive lending algorithms can skew against minority groups simply because there is less data available about them, even if their expected repayment rate is the same. It can be self-reinforcing. Imagine some new demographic group of customers appears, and without any data you make some loans to them. The actual repayment rate will be low, not because that group has a worse distribution than other groups, but simply because you couldn't identify the lowest-risk members. A simplistic ML model would conclude that the new group is more risky. Of course, smart lenders understand that in order…
Read the full comment
to develop a new customer demographic they need to experiment by lending, with the expectation that their first loans will have high losses, but that in the long run learning about how to identify the low-risk people from that demographic is worthwhile. And they correct for the fact that the first cohort was accepted blind when estimating overall risk for the group.
Original comment Approaching fairness in machine learning
wiwofone
Neutral mention · rule
Here is a paper [1] discussing the effects of the new European Union legislation mentioned in the article ("right to explanation"). Interesting related read, regardless of the validity of the numbers mentioned in this opinion piece. [1] http://arxiv.org/pdf/1606.08813v2.pdf
Original comment Make Algorithms Accountable
Smerity
Neutral mention · rule
The European Union are likely to accelerate much of this type of research. They recently introduced regulations specifically targeting algorithmic decision-making and a "right to explanation" for automated decisions. There's a Wired article[1] on the general scene and a paper that investigates the impact it may have on the industry[2]. Regarding differential discrimination, check out these two papers[3][4] - they're very similar to your idea :) [1]: http://www.wired.com/2016/07/artificial-intelligence-setting... [2]: http://arxiv.org/abs/1606.08813v2 [3]: http://arxiv.org/abs/1511.05897 [4]: http://arxiv.org/abs/1412.3756
Original comment Training Computers to Find Future Criminals
Isamu
Neutral mention · rule
From the linked paper (http://arxiv.org/pdf/1606.08813v1.pdf) EU regulations on algorithmic decision-making and a “right to explanation” Bryce Goodman, Seth Flaxman We summarize the potential impact that the European Union’s new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which “significantly affect” users. The law will also create a “right to explanation,” whereby a user can ask for an explanation of an algorithmic decision that was made about…
Read the full comment
them. We argue that while this law will pose large challenges for industry, it highlights opportunities for machine learning researchers to take the lead in designing algorithms and evaluation frameworks which avoid discrimination. From the regulation: Article 11. Automated individual decision making 1. Member States shall provide for a decision based solely on automated processing, including profiling, which produces an adverse legal effect concerning the data subject or significantly affects him or her, to be prohibited unless authorised by Union or Member State law to which the controller is subject and which provides appropriate safeguards for the rights and freedoms of the data subject, at least the right to obtain human intervention on the part of the controller. 2. Decisions referred to in paragraph 1 of this Article shall not be based on special categories of personal data referred to in Article 10, unless suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests are in place. 3. Profiling that results in discrimination against natural persons on the basis of special categories of personal data referred to in Article 10 shall be prohibited, in accordance with Union law.
Original comment Artificial Intelligence Is Setting Up the Internet for a Huge Clash with Europe

Keep following the thread.

More in Algorithms

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score